MICROCOPY  R[  SOI  U I  ION  1 1  ST  CHAR' 

NA”.  N.'.i  •• 


AFHRL-TR-80-32 


AIR  FORCE  SYSTEMS  COMMAND 

BROOKS  AIR  FORCE  BASE, TEXAS  78235 


\oru*K 


^  hrn  l  >  (M^rmnirul  titaniums.  «|r<vttt«*Alioti».  ot  othn  data  air  u*rd  (oi  am  pmjHVxr  othn 
tlu«  a  drlimtrK  ivlatrd  \*orrtnmrni  pnvmrmrnt  opriation.  (hr  liorriumrnt  (hrtrbr  mrui* 
tit*  iv^ponsihiliU  nor  aim  obligation  nhatstvr  m .  ami  thr  (ai  t  (hat  thr  (ioirtmurnl  mar  tuic 
(oimulatrd.  (uinishrd  ot  in  am  war  supphrd  (hr  *anl  tiiaton^,  spcvifiratiom.  01  othri  data 
i*  not  to  hr  tr^airh'd  In  unjOn  atuni  01  t'lhonoM*.  as  in  am  mannrt  hrrnsiujg  (hr-  hoMrt  01  am 
othrr  jrrrson  ot  <vi|HH*tnrn.  ot  rom  rrni£  am  light*  oi  \h-x  motion  (o  mauutarUirr.  %i*r.  oi  >c\\ 
am  patrntrd  imrutton  that  mar  m  am  war  l*r  trlafrti  thrnrlo 

I  his  (mat  ir|vit  suhmittnl  hr  (hr  Man^orrri  and  IVrsoimrl  Ihnsiou.  mulct  I'rojrvt  '  Ml. 
with  11^  \u  Konv  Human  Krsounvs  labntatnti  tVKSt'l.  HronV*  \u  Kmr  Haso.  lY\a> 
?S'.V'  Mi  Komi  Finstiiou  was  tho  I’timipal  Ini  osti^ali'i  fnr  llio  labntatnti 

1  Ins  tvpnit  has  boon  roitouosl  In  tlio  iMlirc  ol  I’libbo  \t fairs  (I’O  ami  is  rrlrasahlr  In  llio 
National  Voolliuoat  liifotmattoii  S'niiT  (N  FIS'  ll  Nl'IS,  il  mil  l>o  aiailahlr  In  llio  |jriirial 
publio.  inoltnltng  I oioi^n  nations 

Vlns  toohnioal  ivporl  lias  boon  iviiomtl  a  ml  is  approi  nl  for  piiblioation 

\  VNtN  t-l  INN.  r<vhnn'*l  lbmvlni 
Manponri  ami  lYrsounol  I'nision 

HON  M  \\  Vk  VFHHN  .  «  olonol.  I  S  W 
('nnimamtoi 


SlBJFtT  l\>  KNIVKl  I'ONIROI  I  l«S 

Vlns  tloounionl  mutants  information  (nr  maiiiifanturiiig  nr  u*tn|t  iiiiinitiniis  nf  »ar  Fvporl  of 
ibo  information  ivnlmnnl  horrm.  nr  rrloaso  In  fnmpii  nationals  mlbin  ihr  l  nitni  Stairs, 
mlbniil  firs!  obtaining  an  ovport  Uoonso .  is  a  Mnlatinn  nf  thr  International  Vraffio  m  Inns 
Regulation*  Snob  nnlalinn  is  subjrol  In  a  (vnalli  nf  up  In  ’2  loans  iiiipnsnnnionl  amt  a  fmr  nf 
$  1 00.000  iimtor  22  1  St  J : 7H 


«"V 


‘jav 


Unclassified 

£uaiT\CLASSIFICATtON  OF  THIS  RAGE  (Whoa  Polo  Entorod) 

Tit]  REPORT  DOCUMENTATION  PAGE 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


|2.  GOV.T  ACCESSION  NO.I  »•  SHCtPIENT’S  CATALOG  NUMBER 


AFHRL-TR-80-32 


693  S' 


4.  TITLE  (and  Subtill*) 

T  management  engineering  team  application  of 

j  OFFICER  GRADE  REQUIREMENTS  METHOD  , 


s5,  TYPEOF  REPORT  A  PERIOD  COVERED 

9~Final  ^  CaSTS 

4.  PERFORMING  ORG.  REPORT  HUMBER 


a  ■  — ...  ““  “ 

Ken  nr  Finstuen 
Gary  N./Matthews 
William  H./Pope  ^ 

.  Performing  organization  name  and  address 
Manpower  and  Personnel  Division 
Air  Force  Human  Resources  Laboratory 
Brooks  Air  Force  Base,  Texas  78235 

t.  CONTROLLING  OFFICE  NAME  ANO  ADDRESS 

HQ  Air  Force  Human  Resources  Laboratory  (AFSC) 
Brooks  Air  Force  Base,  Texas  78235 


T*  MONITORING  AGENCY  NAME  A  ADDRESS (II  dlttotont  from  Controlling  Ottico ) 


-  '  r 

(  \0  '  / 


[  •.  CONTRACT  OR  GRANT  NUMBERf*] 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 


-62703F 

J.».  ;  77346208 

,  - -  « - - - 

^  -REPO M-OATG  ■ 


;/ 


13.  NUMBER  OF  PAGES 

70 

IS.  SECURITY  CLASS.  (ofthta  report) 

Unclassified 


15«.  DECLASSIFICATION/  downgrading 
SCHEDULE 


116.  DISTRIBUTION  STATEMENT  (ol  thl a  Report) 


Approved  for  public  release;  distribution  unlimited. 


[  17.  DISTRIBUTION  STATEMENT  (ol  tho  abatrmct  entered  in  Block  20,  II  dlllarant  from  Report) 


I  18.  SUPPLEMENTARY  NOTES 


1».  KEY  WOROS  (Continue  on  revatae  tide  II  neceaamry  and  Idantlly  by  block  number) 
descriptive  crossbreaks  management  engineering  team 

inter-rater  reliability  multiple  regression  analysis 

job  descriptions  officer  grade  requirements 

job  evaluation  polynomial  fitting 

job  requirement  factore  rating  scales 

20,«^BSTRACT  (Contlnuo  on  rovorao  oido  II  naceaaery  and  Idantlly  by  block  numbar) 

“7  This  report  presents  detailed  results  of  a  large  scale  Management  Engineering  Team  (MET)  application  of  the 
Officer  Grade  Requirements  (OCR)  methodology.  The  OCR  technology  was  applied  to  determine  the  appropriate 
grades  for  non-aircTew  Air  Force  officer  positions  based  on  job  content  and  responsibility.  Based  on  data  from  over 
1 1,000  position  descriptions  collected  by  METs  and  MET  ratings  of  a  representative  sample  of  descriptions  from  the 
original  OCR  project  the  following  major  steps  were  accomplished:  (a?  development  of  an  efficient  linear  regression 
equation  for  grade  determination,  -(h)  construction  of  a  stable  grade  conversion  table,  and  -(^computation  of  validity 
and  reliability  information  concerning  MET  rating  behavior.  These  analyses  indicated  that  METs  apply  the 
technology  in  a  systematic  and  reliable  manner,  resulting  in  a  defensible  statement  of  the  appropriate  military  grade 
for  a  given  non-aircrew  job. 


oo  1473 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  RAGE  D*f*  Entered) 


SUMMARY 


r '  y 


/ 


Objective 

The  primary  objectives  of  this  report  are  to  provide  (a)  an  overview  of  a  large  scale  Management 
Engineering  Team  (MET)  field  test  of  the  officer  grade  requirements  (OCR)  technology,  (b)  a 
comprehensive  description  of  the  development  of  the  job  evaluation  method  by  which  grades  may  be 
determined,  (c)  a  detailed  report  of  results  and  findings  from  the  field  test,  and  (d)  a  discussion  of  the 
implication  of  these  findings  for  operational  use  by  METs. 

This  document  is  a  companion  report  to  AFHRL-TR-80-31,  which  presents  an  historical  overview  of 
the  OCR  project— 1963  to  present— inchtdmg  aTislirig  of  the  grade  structure  projected  to  54  career 
utilization  fields  and  the  entire  non-aircrew  officer  force.  This  report  provides  documentation  of  the  job 
evaluation  methodology  and  data  analysis  and  supplements  the  preceding  report. 

Approach 

The  OCR  technology  has  been  under  development  at  various  times  for  over  15  years.  During  this 
present  phase  of  the  research  effort,  the  OGR  method  was  adapted  and  applied  by  METs  in  a  large-scale 
field  test.  This  report  provides  extensive  analyses  of  the  reliabilities  and  validities  of  measurements,  as 
well  as  an  explanation  of  the  development  of  an  operational  grade  determination  equation  and  the 
construction  of  a  stable  grade  conversion  table. 

Specifics 

Using  the  basic  technology  developed  >n  a  preliminary  feasibility  study  (AFHRL-TR-75-80),  the 
OGR  method  was  applied  to  over  11,000  current  Air  Force  officer  job  descriptions  to  determine  the 
appropriate  grades  for  those  positions,  based  on  job  content  and  responsibility.  METs  collected  and  rated 
the  current  job  descriptions,  as  well  as  a  representative  sample  of  descriptions  of  jobs  for  which  grades  had 
been  determined  by  an  Air  Force  Policy  Board  in  the  original  OCR  project.  Job  ratings  employing 
benchmark  scale  rating  techniques  were  made  on  job  evaluation  factors. 

Once  the  initial  data  collection  from  the  field  was  complete,  a  comprehensive  set  of  reliability  of 
measurement  coefficients  were  computed  for  both  types  of  ratings  to  ensure  that  all  MET  members  were 
rating  jobs  in  a  similar  fashion.  Ratings  on  various  job  factors  together  with  position  description  data  were 
then  entered  into  an  eight-variable  multiple  linear  regression  equation  to  predict  the  appropriate  grade 
for  a  given  job.  The  predictors  in  the  final  equation  were  (a)  special  training  and  work  experience,  (b) 
communication  skills,  (c)  judgment  and  decision  making,  (d)  planning,  (e)  management,  (f)  level  of 
organization,  (g)  level  of  job  within  organization,  and  (h)  the  supervisor’s  judgment  of  the  appropriate 
grade  for  the  job. 

Taking  job  composite  scores  generated  by  the  grade  determination  equation,  a  stable  grade 
conversion  table  was  then  constructed  for  research  purposes  which  converted  weighted  job  composite 
scores  into  appropriate  military  grades  from  Lieutenant  through  Colonel.  Details  for  the  construction  of  a 
conversion  table  which  would  reflect  present  career  progression  policy  are  given  also. 

Conclusions 

The  results  presented  in  this  report  indicate  that  there  was  a  high  agreement  among  MET  raters  as  to 
the  job  content  and  responsibility  levels  of  jobs.  Rating  reliabilities  are  reported  by  rater  type  (officer, 
civilian,  and  enlisted  MET  members)  and  for  all  job  evaluation  factors  used  in  the  field  test  and  indicate 
high  inter-rater  agreement  among  raters. 
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METs  were  able  to  accurately  apply  the  technology  based  on  validity  compulations.  The  validity  of 
the  grade  equation  compared  to  judgments  of  grades  made  by  an  Air  Force  Policy  Board  resulted  in  a 
multiple  regression  coefficient  of  .90,  indicating  a  high  predirtive  efficiency.  In  addition  to  the  final  grade 
composite  validation,  computations  of  validities  for  the  eight  grade-determination  equation  components 
are  also  reported  and  indicate  the  system  assigns  grades  in  a  fashion  consistent  with  judgments  made  by 
the  Air  Force  Policy  Board. 

In  addition  to  the  aforementioned  data  analyses,  results,  and  procedure  descriptions,  the  report 
provides  samples  of  all  data  collection  forms  and  rating  instruments  developed  in  the  study. 
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PREFACE 


This  research  was  completed  under  Project  7734,  Development  of  Methods  for  Describing, 
Evaluating,  and  Structuring  Air  Force  Occupations;  Work  Unit  77340208,  Determination  of  Non- 
Aircrew  Officer  Grade  Requirements:  Conversion  Tables  and  Projections  by  Management 
Engineering  Teams.  The  work  unit  was  established  in  response  to  a  Request  for  Personnel  Research 
(RPR  75-21)  submitted  by  HQ  USAF/PRMRE  and  DPXX  titled  “Determination  of  Officer  Grade 
Requirements  Based  on  job  Content  and  Responsibilities.”  The  RPR  requested  that  the  initial  test 
(Stacy,  Matthews,  &  Hazel,  1975)  of  the  Management  Engineering  Team  (MET)  job  rating  and 
evaluation  system  be  further  developed  and  expanded  to  a  functional  technology  which  could  be 
implemented  by  the  METs.  The  request  specifically  entailed  (a)  extension  of  the  previous  research 
in  developing  a  reliable  and  svstematic  method  the  METs  could  use  to  determine  appropriate  grades 
for  jobs  based  on  their  content  and  responsibility  levels  and  (b)  application  of  this  technology  to 
forecast  the  non-aircrew  officer  grade  requirements  in  an  empirical  manner. 

The  Air  Force  Management  Engineering  Agency  (AFMEA)  served  as  the  monitoring  agency 
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Project  Monitor. 
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This  study  does  not  constitute  authority  to  change  existing  Air  Force  officer  grades,  and  its 
publication  does  not  infer  approval  to  implement  report  procedures  as  USAF  policy.  Such  actions  or 
objectives  are  a  function  of  HQ  USAF.  Further,  the  report  does  not  attempt  to  tie  into  or  address 
provisions  of  the  Defense  Officer  Personnel  Management  Act  (DOPMA)  nor  claim  compatibility 
with  same.  While  the  report  does  briefly  mention  grade  authorizations  and  career  planning 
objectives,  it  neither  fully  addresses  nor  encompasses  the  entire  problem  of  grade  distributions  to 
support  orderly  career  progression  plans,  nor  recognizes  external  limitations  on  Air  Force  grade 
structure.  Although  the  philosophy  and  methodology  of  the  report  are  assumed  to  be  valid,  accurate 
and  unbiased,  the  report  should  be  viewed  from  the  prospective  that  considerable  efforts  over  the 
years  have  allowed  USAF  to  determine,  establish,  and  defend  Air  Force  grade  requirements,  albeit 
multi-faceted  approaches  were  utilized. 
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MANAGEMENT  ENGINEERING  TEAM  APPLICATION 
OF  OFFICER  GRADE  REQUIREMENTS  METHOD 


I.  INTRODUCTION 

The  methodology  and  various  technical  details  involved  in  conducting  research  to  determine  non¬ 
aircrew  officer  grade  requirements  based  on  job  content  and  responsibility  is  presented  in  this  report, 
which  is  a  companion  publication  to  AFHRL-TR-80-31  (Hazel  4  Finstuen,  1980).  This  report  extends  the 
major  findings  and  results  which  were  highlighted  in  the  previous  report,  and  discusses  the  techniques 
used  to  arrive  at  those  findings.  More  specifically,  this  report  elaborates  on  such  items  as  (a)  the  various 
sample  size  sets  and  subsets  used  during  the  entire  stream  of  Officer  Grade  Requirement  (OGR)  research, 
(b)  the  development  of  factors  and  weights  used  in  the  OGR  eight-variable  equation,  (c)  the  inter-rater 
reliabilities  computed  for  various  sample  sets  and  subsets  of  jobs  judged  by  various  types  of  Management 
Engineering  Team  (MET)  raters,  (d)  validity  comparisons  of  the  final  regression  equation  used,  and  (e) 
the  conversion  table  and  how  it  was  generated  and  applied  during  this  cycle  of  the  OGR  research. 

The  first  OGR  system  of  grade  determination,  commencing  in  1963,  used  field  grade  officers  as  raters 
(Hazel.  1965;  Hoggatt  4  Christal.  1966).  The  grade  determination  process  was  applied  to  the  total  officer 
force  in  1964  using  a  projection  technique  (Christal,  1965).  This  allowed  a  comparison  between  the 
existing  Unit  Manning  Document  (UMD — now  termed  UDL,  Unit  Detail  Listing)  statements  of  needed 
grades  for  various  officer  career  specialties  and  the  OCR  equation  statements  of  appropriate  military 
grade  for  the  same  jobs.  A  small-scale  test  of  the  system  using  MET  raters  was  later  carried  out  in  1974 
(Stacy.  Matthews.  &  Hazel,  1975),  indicating  the  feasibility  of  a  large-scale  field  test  of  the  technology. 

The  end  product  of  the  present  OGR  research  resulted  in  an  efficient  officer  grade  determination 
system  which  assigns  grades  to  non-aircrew  officer  jobs.  The  essential  components  of  this  system  consist  of 
a  set  of  factors  and  job  attributes  which,  when  entered  into  a  regression  equation,  are  used  to  determine 
job  content  and  responsibility  scores.  These  predicted  composite  scores  for  any  judged  job  are  then 
converted  to  an  appropriate  military  grade.  The  1976  OGR  study  updated  projections  of  appropriate  grade 
requirements  for  over  60,000  non-aircrew  officer  positions. 


A  History  of  Sample  Sets  (1963—1977) 

The  history  of  the  OGR  research  has  been  presented  at  length  in  several  technical  reports  (Christal. 
1965  4  1975;  Hazel,  1965;  Hoggatt  4  Christal.  1966;  et  al„  1975).  A  review  of  events  occurring  in  the 
stream  of  research  is  given  in  Table  1. 


Table  1.  History  of  Officer  Grade  Requirements  (OGR)  Research 


Year 

OGR  B>ent 

1963-1966 

Air  Force  Policy  Board 

Policy-Capturing  Equations  Developed 

Initial  Projections  Made  for  Total  Air  Force 

1966 

Development  of  Benchmark  Scales 

1974-1975 

Pilot  Test  of  Management  Engineering  Team 

Application  of  Grade  Determination  Procedure 

1976-1977 

Large-Scale  Field  Test  of  the  Technology 
and  Construction  of  a  Grade  Conversion  Table 

Projections  to  Non-Aircrew  Force 
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At  each  of  the  stages  of  OGR  development,  samples  (of  jobs)  have  been  drawn  from  various  sets  and 
subsets  of  job  pools.  In  presenting  the  history  of  the  research,  the  job  sample  sets  would  be  useful  both  in 
an  e\p«  itory  and  an  analytic  presentation.  This  report  analyzes  each  of  the  OGR  events  in  terms  of  the 
number  of  job  samples  (n)  used  for  each  stage  of  the  research.  The  initial  Policy  hoard  sample  jobs  (N  = 
3,575)  were  extracted  from  a  collection  of  approximately  80,000  job  descriptions  gathered  in  1963  (Hazel, 
1065;  Hoggatt  &  Christa).  1066).  These  3.575  jobs  were  evaluated  by  an  Air  Force  Policy  Board  comprised 
of  22  colonels,  and  the  first  OGR  policy-capturing  equation  was  developed  from  this  sample.  Of  these  jobs, 
1,000  were  selected  for  the  creation  of  a  system  of  benchmark  scales  developed  in  1966  (Brokaw  & 
Giorgia).  Using  benchmark  scales,  it  was  found  that  ratings  provided  by  field  raters,  when  entered  into  an 
equation,  generated  grades  which  were  nearly  identical  with  those  assigned  by  the  Policy  Board  (r  =  .90). 
The  initial  parent  population  and  subsequent  samples  drawn  from  it  are  depicted  in  Figure  1. 


1963  Job 
Collection 
N  -  79,750 


1966 

Benchmark  Jobs 


Figure  1.  Sample  seta  of  jobs. 


During  the  1963-1966  period  (see  Table  1),  grades  for  a  subset  of  8.250  jobs  from  the  original 
79.750  were  computed  using  the  OGR  technology  and  were  combined  with  the  sample  of  3,575  Policy 
Board  jobs  resulting  in  a  set  of  1 1,825  (3,575  +  8,250)  jobs  (Hoggatt  &  Christal.  1966).  This  sample  set 
became  the  projection  base  which  was  used  in  determining  appropriate  officer  grade  requirements  for  all 
Air  Force  officer  jobs  in  1964.  In  the  more  recent  1974  pilot  study,  485  of  the  3,575  Policy  Board  jobs  were 
rated  again,  this  time  using  MET  personnel,  rather  than  field  grade  officers  (Stacy  et  al„  1975),  as  raters. 
In  addition.  1.687  current  job  descriptions  were  also  collected  and  rated,  resulting  in  2,172  jobs  being 
rated  in  the  1974  pilot  study.  For  clarification,  these  N  sets  are  arrayed  in  descriptive  crossbreak  form  with 
other  N  sets  in  Table  2  (Kerlinger.  1973). 
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The  pilot  study  proved  successful  and  indicated  that  MET  members  could  apply  the  benchmark 
rating  scales  in  the  same  manner  that  field  grade  officer  raters  had  and  that  there  was  high  inter-rater 
agreement  among  the  MET  members  as  to  discernible  levels  of  job  content  and  responsibility. 

An  expansion  of  the  pilot  study  was  undertaken  to  subject  the  technology  to  a  full  field  test  using 
METs.  As  shown  in  Figure  1  and  Table  2,  an  additional  1,240  Policy  Board  jobs  were  rated  and  combined 
with  the  485  pilot  study  jobs.  This  resulted  in  a  1,725  job  set  for  computing  validity  (comparing  grades 
assigned  by  METs  with  those  assigned  by  the  Policy  Board)  and  for  computing  a  grade  conversion  table.  In 
addition,  9,634  current  jobs  were  collected,  rated  and  added  to  the  1 ,687  current  jobs  from  the  pilot  study 
to  provide  a  base  of  over  11,000  jobs  to  use  in  making  projections  of  officer  grade  requirements  to  the 
entire  non-aircrew  force  (Hazel  &  Finstuen,  1980). 


Table  2.  Sample  Crossbreak  of  Subsets  and  Total 
Number  of  Jobs  Used  in  Job  Evaluation 


Policy 

Board  Jobs 

Current 

Jobs 

Totals 

1974  Pilot 

Study 

N  =  485  (1) 

N= 1,687  (4) 

N=2,172  (7) 

1976  Main 

Study 

N= 1,240  (2) 

N=9,634  (5) 

N=  10,874  (8) 

Total 

Sample 

N  =  1,725  (3) 

N= 1 1 ,321  (6) 

N= 13,046  (9) 

Cells 

Description 

(1)  to  (3) 

Policy  Board  jobs  used  as  the  criterion  in  computing 
validity  and  construction  of  the  grade  conversion  table 

(4)  to  (6) 

Current  jobs  newly  collected  used  as  the  base  for 
making  projections  to  the  total  non-aircrew  force 

(7)  and  (8) 

Rating  subset  information  used  for  reliability 
computations 

(9) 

Crand  total  number  of  all  jobs  used  in  the  study 

The  job  evaluation  effort  of  the  OCR  studies  has  resulted  in  ratings  and  computations  of 
recommended  grades  for  more  than  23,000  jobs  with  projections  addressing  176,000  officer  jobs  at 
various  points  in  time  over  the  past  14  years.  As  a  result,  the  OGR  is  perhaps  the  largest  and  most 
comprehensive  job  evaluation  study  of  its  kind  in  existence. 


II.  METHODOLOGY 


Gn  a.c  Determination  Process.  The  Air  Force  Human  Resources  Laboratory  (AFHRl.)  had  at  it* 
disposal  a  long  and  well-documented  research  effort  to  draw  upon,  having  developed  factors  and  scales  for 
use  in  previous  studies.  The  essential  elements  for  job  evaluation  consist  of  jobs,  job  factors,  a  systematic 
evaluation  scheme  and  a  procedure  which  can  rank-order  a  given  set  of  jobs.  These  elements  were 
incorporated  into  the  OCR  job  evaluation  method,  each  element  being  tailored  for  use  specifically  in  an 
Air  Force  setting.  The  design  of  the  present  field  test  followed  the  previous  grade  determination  process 
which  had  been  developed.  Figure  2  depicts  the  grade  determination  process. 


Figure  2.  The  grade  determination  process. 


Statistical  Considerations.  In  order  to  assess  the  dependability  of  MET  raters'  judgments,  compared 
to  the  judgments  of  the  Policy  Board,  MET  raters  provided  factor  ratings  on  a  subset  of  1.725  positions 
from  the  original  criterion  job  set  of  3,575  jobs.  Scores  resulting  from  the  application  of  the  grade 
determination  equation  can  thus  be  compared  with  the  grades  assigned  by  the  Air  Force  Policy  Board. 
This  comparison  provides  an  index  to  the  validity  of  the  MET  ratings.  A  high  coefficient  would  assure  that 
ratings  made  by  MET  members  can  be  applied  to  individual  jobs  to  determine  the  appropriate  grade  based 
on  the  content  and  the  responsibility  of  the  position.  Portions  of  the  1,725-job  set  were  mailed  to  nearly  all 
METs  participating  in  the  survey,  and  they  rated  the  jobs  using  the  benchmark  scale  technique. 

In  addition,  the  sample  of  1 1,000  current  jobs  was  used  to  evaluate  MET  rating  agreement.  This  same 
sample  of  current  jobs  was  also  used  as  the  basis  for  making  projections.  A  high  correlation  or  agreement 
among  MET  raters  provides  an  index  to  the  reliability  of  the  technology. 

The  grade  conversion  table  was  constructed  from  the  1 ,725  job  sample.  Essentiallv  it  was  constructed 
in  such  a  manner  that  the  METs  assigned  the  same  number  of  jobs  to  a  grade  level  as  were  previously 
assigned  to  that  grade  level  by  the  1964  Policy  Board.  (Christal,  1965.  referred  to  this  process  as  an  equi- 
percentile  conversion). 

After  all  current  jobs  had  been  computed  and  converted,  the  1 1 ,000  job  set  was  used  as  the  base  for 
estimating  the  appropriate  grades  for  the  total  non-aircrew  force  This  was  accomplished  for  each  of  the 


10 


Air  Force  Specially  Codes  (AFSCs)  representing  54  officer  utilization  fields.  The  process  involved 
assigning  or  projecting  sample  OCR  changes  of  grade  to  the  remaining  non-aircrew  AFSC  jobs  from  which 
the  sample  jobs  were  drawn. 

OCR  projected  grades  were  compared  with  the  UDL  grades  assigned  to  jobs  to  assess  the  impact  of  the 
technology.  Existing  grade  structures  were  also  reported  from  the  on-board  strength  figures  in  the 
Uniform  Officer  Records  (UORs).  Since  projections  can  be  only  as  definitive  and  usable  as  the  sample  on 
which  they  are  based,  those  career  areas  where  sample  size  was  a  limiting  factor  for  grade  determination 
included  a  cautionary  statement  to  preclude  misapplication  or  misinterpretation.  The  projections  and 
various  comparisons  are  reported  in  Appendix  C  of  Hazel  and  Finstuen  (1980). 


Measurement  Procedures  and  Instructions 

Instruments  used  in  the  study  were  (a)  officer  position  descriptions  (Appendix  A),  (b)  MET  rater 
background  and  rating  forms  (Appendix  B),  (c)  the  benchmark  scales  used  in  both  the  pilot  study  and  the 
field  test  (Appendix  C).  and  (d)  the  Policy  Board  16-point  grade  code  scale  (Appendix  D). 

From  November  1975  to  March  1976,  officer  position  descriptions  were  filled  out  by  the  incumbents, 
stating  the  tasks  and  duties  that  were  assigned  to  them  as  integral  parts  of  their  work.  The  incumbent  then 
forwarded  the  description  to  the  supervisor  who  made  a  judgment  as  to  the  appropriate  grade  for  the  job. 
The  supervisor  then  returned  the  position  description  to  the  MET.  At  the  direction  of  the  MET  project 
officer.  MET  monitors  then  fastened  the  position  descriptions  into  folders  with  25  randomly  sorted 
descriptions  per  folder.  (See  Project  Officer  Instructions  — Appendix  E.)  Descriptions  were  randomly 
assigned  to  folders  to  control  for  contextual  rating  effects  (Madden,  1960).  Descriptions  were  bound  so 
that  the  MET  rater  could  read  only  pages  2  and  3  which  contained  task  and  duty  information.  MET  raters 
employed  the  benchmark  scales  in  matching  a  job  to  a  level  of  content  and  responsibility  for  each  of  the 
10  grade  defining  factors  and  recorded  their  responses  on  the  rating  data  sheet  (Appendix  B).  They  also 
nrovided  their  judgment  of  the  appropriate  grade  using  the  16-point  Policy  Board  grade  scale.  Rating  in 
this  manner,  the  MET  rater  had  no  knowledge  of  the  actual  grade  of  the  incumbent,  the  UDL  grade  for  the 
job.  nor  the  supervisor’s  judgment  of  the  job’s  grade. 

In  addition.  MET  raters  provided  both  benchmark  scaled  judgments  for  factors  and  16-point  grade 
scale  judgments  for  original  Policy  Board  jobs.  These  job  descriptions,  used  for  the  validation  procedure, 
had  been  randomly  sorted  and  bound  into  folders  in  a  manner  similar  to  those  for  current  jobs.  All  folders 
were  then  checked  for  completeness  after  being  rated  and  were  forwarded  to  AFHRL. 

Previous  inter-rater  agreement  of  MET  members  using  the  benchmark  scales  during  the  pilot  study 
had  resulted  in  composite  score  reliabilities  of  .94  and  .95  for  Policy  Board  and  current  jobs,  respectively. 
Reliabilities  using  the  16-point  Policy  Board  grade  scale  were  .90  for  Policy  Board  and  .93  for  current  jobs 
(Stacy  et  al.,  1975).  Based  on  this  high  degree  of  reliability,  it  was  expected  that  measurements  made  with 
these  instruments  (the  existing  job  description  forms  and  benchmark  scales)  for  the  main  field  test  would 
be  of  comparable  quality. 


Job  Sample  Sets 

The  entire  study  consisted  of  two  central  sample  sets  (see  Table  2).  The  first,  or  mail-out  sample  of 
1.725  Policy  Board  jobs  was  taken  from  utilization  fields  which  were  still  in  the  Air  Force  inventory  of 
specialties  and  were  essentially  unchanged  since  1964.  All  major  commands  were  represented  in  the 
sample.  (In  Appendix  A,  see  Page  4  of  the  job  description  for  the  list  of  2)  major  commands.)  The  sample 
was  stratified  across  all  UMD  authorized  grades  from  lieutenant  to  colonel  at  various  levels  of  organization 
ranging  from  Detachment  and  Squadron  levels  up  to  Hq  USAF.  The  1 .725  Policy  Board  sample  consisted 
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of  two  subsets.  The  485-job  subset  had  been  rated  in  the  pilot  study  in  1974.  The  1,240-job  subset  used  in 
this  field  study  was  rated  in  an  identical  manner  as  the  485-job  subset.  Two  copies  of  the  job  descriptions 
were  made.  Job  descriptions  were  randomly  sorted  and  bound  into  folders,  25  per  folder.  Two  copies  of 
each  job  description  were  made  so  that  the  same  job  would  be  rated  by  two  different  METs.  This  allowed  a 
comparison  of  METs'  rating  behavior.  There  was  little  difference  between  the  way  METs  viewed  jobs  as 
reported  earlier  by  Stacy  et  al.,  1975. 

A  second,  current,  sample  set  consisting  of  over  11,000  jobs  was  collected  during  the  pilot  and  main 
studies.  During  the  main  study  development,  current  jobs  were  stratified  across  grade  and  Duty  AFSC  to 
assure  that  positions  selected  by  the  METs  in  the  field  were  representative  of  the  non-aircrew  officer 
population  to  which  the  technology  would  be  applied.  The  1975  UDL  of  Air  Force  jobs  was  used  to  specify 
which  grade  within  AFSC  should  be  selected  by  the  MET  as  a  category  for  sampling.  In  all,  11,137  jobs 
were  specified  as  potential  samples  in  53  career  areas  across  all  grades. 

METs  were  instructed  to  randomly  sort  and  bind  the  new  job  descriptions  which  they  collected,  with 
twenty-five  descriptions  to  a  folder. 


Data  Collection  and  MET  Raters 

The  samples  were  taken  using  122  METs  (see  Appendix  F)  consisting  of  954  raters  covering  all  Air 
Force  commands  both  overseas  and  within  the  continental  U.S.  during  1974 — 1976.  Ratings  were  obtained 
for  the  1,240  Policy  Board  job  descriptions  which  had  been  mailed  out,  and  for  9,634  which  METs  had 
gathered  based  on  the  preselected  grade  by  AFSC  classification  sample  specification.  METs  then  mailed  all 
materials,  including  the  job  descriptions,  ratings,  and  rater  background  information  to  AFHRL.  Upon 
receipt,  ratings  obtained  on  the  1,240  Policy  Board  jobs  were  merged  with  the  485  job  ratings  from  the 
pilot  study  after  screening  for  completeness.  Only  one  of  the  106  METs  which  received  Policy  Board  jobs 
failed  to  return  ratings. 

Likewise,  the  returned  9,909  current  jobs  which  had  been  collected  from  the  11,137  specified  (89% 
returned)  were  screened  for  rating  completeness.  Of  these,  275  jobs  were  eliminated  due  to  too  few 
ratings,  inattentive  or  uncooperative  raters,  or  illegible  position  numbers,  leaving  9,634  jobs  which  were 
merged  with  the  1,687  current  jobs  from  the  pilot  study.  This  resulted  in  job  ratings  for  11,321  jobs  used 
for  computing  rater  reliabilities.  Of  those  11,321  jobs,  129  did  not  contain  UDL  grade  information 
essential  for  computing  projections.  The  129  jobs  were  included  in  reliability  computations  and  were 
removed  for  subsequent  projection  application.  The  projection  job  set,  cutting  across  54  career  areas  by 
grade  is  presented  in  Appendix  G. 

An  average  of  13.61  rating  judgments  were  provided  on  job  factors  based  on  the  benchmark  scales 
for  each  position  in  the  1,725  job  sample.  The  new  1 1,321  current  jobs  reflected  an  average  of  6.95  MET 
ratings  for  each  job  considered.  In  all,  13,046  non-aircrew  officer  position  descriptions  were  judged  on  10 
grade  factors  and  a  16-point  grade  scale  resulting  in  over  143,000  separate  job  evaluation  judgments 
obtained  from  954  trained  MET  raters.  Means  and  standard  deviations  for  the  factors  and  scale  are 
presented  in  Appendix  H. 

Of  the  954  Air  Force  rater  personnel,  591  indicated  they  were  enlisted  E-4  to  E-9,  175  were  officers 
(0-1  to  0-5),  184  checked  civilian  (CS-5  to  CS-14),  and  four  failed  to  respond  to  this  background 
information  item.  Years  of  experience  in  management  engineering  or  manpower  ranged  from  1  to  28 
years,  with  an  average  of  7  years  2  months  of  experience  for  the  typical  rater.  Over  82%  of  the  raters 
responding  had  completed  either  the  Air  Force  Management  Engineering  course  or  the  urmy 
Management  Engineering  Training  Agency’s  work  methods  and  standards  course.  All  major  commands 
listed  on  the  rater  form  were  represented.  Of  the  larger  percentages  reported  by  raters,  21%  indicated  they 
were  assigned  to  a  Strategic  Air  Command  (SAC)  unit,  12%  to  a  Tactical  Air  Command  (TAC)  unit,  and 
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11%  to  Air  Training  Command  (ATC)  installations.  The  number  of  raters  at  eaeh  MET  ranged  from  5  to 
30.  with  an  average  of  7.7  raters  per  MET  (see  Appendix  F).  Based  on  these  qualifications,  the  MET  raters 
were  well-trained  and  experienced,  and  a  good  mix  of  MET  installations,  representative  of  the  MET  force 
in  the  field,  was  used  in  the  study. 


III.  RESULTS  AND  DISCUSSION 

The  data  analyses  employed  in  this  study  consisted  of  development  of  a  grade-determining  equation, 
validity  calculations,  construction  of  a  grade  conversion  table,  and  computations  of  inter-rater  reliabilities 
for  sample  sets.  Each  phase  of  the  data  analysis  is  presented  below  as  a  separate  section. 


Development  of  the  Regression  Equation  (N  =  1,725  Policy  Board  Jobs) 

The  equation  used  in  the  OCR  technology  consists  of  eight  variables  which  renders  a  weighted 
cunpoeite  score  based  on  five  grade-defining  factors  (special  training,  communication,  decision  making, 
planning,  and  management),  two  levels  of  organization  information  (level  of  organization,  level  of  job 
within  organization),  and  the  job  incumbent's  supervisor's  judgment  of  the  appropriate  grade  for  the  job. 
These  variables  were  isolated  in  previous  studies  (Christa),  1965;  Hazel,  Christal.  &  Hoggatt.  1966;  Stacy 
et  al.,  1975)  and  were  found  to  be  stable  grade-defining  variables.  Though  jobs  were  rated  in  terms  of  10 
factors  by  METs  (Appendix  C),  a  regression  analysis  indicated  that  these  five  were  more  definitive  of 
grade. 

Previously,  the  grade  judgment  by  the  MET  (or  rater)  on  the  16-poinl  scale  (Appendix  D)  had  been 
entered  into  the  equation  as  a  variable.  Since  this  study  constituted  the  final  research  test  of  the 
technology  before  it  would  be  submitted  for  possible  implementation,  this  variable  was  removed  from  the 
equation  and  a  new  equation  was  recomputed  and  evaluated.  This  was  done  because  the  16-point  scale 
mean  value  assigned  by  the  MET  raters  to  1,725  jobs  was  6.9  (see  Appendix  H)  or  a  composite  score 
equivalent  to  borderline  low  major.  Yet,  the  average  composite  score  rendered  by  the  equation  would  be 
39.0  which,  when  converted,  centers  on  middle  major,  creating  a  disparity  of  about  nine  composite  points. 
This  shows  that  MET  raters  tended  to  slightly  deflate  their  judgments  of  grade  when  making  a  judgment 
on  the  16-point  Policy  Board  scale,  but  when  using  the  benchmark  scales,  their  judgments,  tempered  with 
other  equation  variables,  were  slightly  higher.  Such  a  condition  could  easily  lead  to  an  inflationary 
tendency  on  the  part  of  a  rater  if  the  appropriate  grade  assigned  to  a  position  by  the  equation  is  always 
slightly  above  the  judged  grade.  In  the  operational  use  of  the  technology,  the  raters  would  attempt  to  bring 
their  judgments  in  line  with  the  grade  assigned  by  the  technology.  In  doing  so,  every  time  they  increased 
their  grade  judgment  value,  the  equation  would  again  adjust  the  assigned  grade  level  upward  by  several 
points,  resulting  in  an  inflationary  trend. 

Regression  analysis,  using  the  Policy  Board  decisions  as  the  criterion,  and  with  five  factors,  two  levels 
of  organization  information,  and  the  supervisor's  judgment  of  the  appropriate  grade  used  as  predictors, 
produced  the  validity  coefficient  R  »  .90.  The  multiple  regression  grade  prediction  equation  may  be 
expressed  in  general  linear  model  form  (Bottenberg  A  Ward,  1963;  Ward  A  Jennings,  1973)  as  shown  in 
formula  1; 

Y’  -  f,F(1)  +  f2F<2)  +f3F(3)  +  f4F(4>  +  f5F(5)  + 

1|L(1)  +  12L(2)  + 

IlC(l)  +  g2G<2>  +  g3G<3)  +  g4G<4>  +  g5G<5>  +  c  (1) 

where  Y'  is  a  predicted  composite  score,  and  the  weights  and  predictor  variables  associated  with  each  score 
are  as  shown  in  Table  3. 
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Table  3.  Regression  Weights  and  Variables  for  OCR  Iquation 


Wetfhu 

Predictor  Variable* 

Sya*ol 

Raw 

Integer 

Symbol 

Description 

*1 

.166 

i 

Factor  2-Special  training  and  work 
experience 

h 

.138 

i 

p(2) 

Factor  5-Communication  skills 

f3 

.089 

l 

p<3) 

Factor  7-judgment  and  decision  making 

f4 

.129 

l 

p(4) 

Factor  8-Planning 

'5 

.459 

3 

p(5) 

Factor  9-Management 

>1 

.118 

1 

L<» 

Level  of  organization 

l2 

.205 

1 

l<2) 

Level  of  job  within  organization 

*1 

-.827 

-12 

c<‘> 

1  if  Lieutenant,  0  otherwise 

*2 

-.533 

-  9 

g<2> 

1  if  Captain.  0  otherwise 

*3 

.000 

-  5 

g<3> 

1  if  Major.  0  otherwise 

84 

1.424 

9 

C<4> 

1  if  Lt  Colonel.  0  otherwise 

85 

3.167 

12 

g<5) 

1  if  Colonel.  0  otherwise 

0 

1.335 

(Regression  constant) 

Correlations  from  the  factors  and  variables  in  the  study  were  analysed  with  a  triple  correcting  regression 
routine  (Ward,  Buchhorn,  &  Hall,  1967,  Ward,  Hall,  A  Buchhorn,  1967)  which  resulted  in  the  reported 
raw  weights  for  the  variables.  For  simplicity  in  application  of  the  technology,  the  raw  weights  were 
adjusted  and  simplified,  converting  the  weight  into  an  integer  value  rather  than  using  the  decimal  weight. 
Although  the  result  was  some  loss  of  predictive  efficiency  (R  =  ,8989  to  R  =  .8958),  the  loss  is  more 
than  compensated  for  by  the  resulting  simplicity  in  computation.  The  loss  in  predictive  efficiency  is  slight 
when  compared  across  R2  or  percent  of  efficiency  of  prediction  dimensions.  When  the  former  equation 
value  R  *  .8989  is  squared,  it  constitutes  an  R2  value  of  .8080  or  80.80%.  An  R2  value  (.8025)  for  the 
simplified  equation  represents  80.25%  or  a  loss  of  about  .5%  in  predictive  efficiency.  A  loss  of  .0055  from 
an  R*  based  on  1,725  jobs,  though  statistically  significant  due  to  such  large  sample  size,  is  of  such  small 
magnitude  in  practical  application  terms  as  to  be  inconsequential  in  the  validation  of  the  method.  The 
simplified  computation  uses  whole  number  weights,  and  in  the  rase  of  multiplying  by  I .  uses  the  mean 
value  for  factor  ratings  and  the  reported  value  for  level  of  organization  variables. 


Validity  (N  *  1,725  Policy  Board  Jobs) 

Table  4  presents  validities  of  variables  included  in  the  MET-applied  simplified  integer  equation 
based  on  13.61  ratings.  As  shown  on  the  table,  the  validity  of  the  final  weighted  composite  is  .90  against 
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the  original  Policy  Board  grade  evaluations,  a  very  high  index  of  concurrent  agreement.  This  value 
indicates  that  the  grade  determination  equation  can  be  applied  in  a  consistent  and  systematic  manner 
using  MKT  rater  judgments  about  non-aircrew  officer  jobs. 


Table  4.  Validities  of  Variables  in  MET  Applied 
Integer  Weight  Grade  Equation  for  N  =  1,725  Jobs 


Predictor  Variable  Validity 


factor  2  -  Special  Training  and  Work  Experience  .65 

Factor  5  -  Communication  .72 

Factor  7  -  Judgment  and  Derision  Making  .74 

Factor  8  -  Planning  .78 

Factor  9  -  Management  .79 

Level  of  Organization  .50 

Level  of  Job  Within  Organization  .47 

Supervisor's  Judgment  of  Grade  for  Job  .78 

Final  tirade  Evaluation  Composite  Score  .90 


Construction  of  the  Research  Conversion  Table  (N  =  1,725  Jobs) 

In  addition  to  the  equation  being  developed,  the  predicted  composite  scores  were  computed  and 
arrayed  in  order  from  the  highest  composite  score  of  84.0  to  the  lowest  score  of  5.1  for  jobs  in  the  1.725  job 
set.  To  establish  a  link  between  the  composite  score  value  and  the  recommended  military  grade  associated 
with  it.  (colonel,  lieutenant  colonel,  major,  captain,  or  lieutenant),  the  following  procedure  was  employed. 

he  "umber  of  jobs  identified  by  the  Policy  Board  for  any  given  grade  were  counted:  for  example,  in  the 
case  of  colonel,  there  were  142  jobs.  Since  the  composite  scores  are  ranked  from  high  to  low  the  top  142 
composite  scores  are  equivalent  with  the  grade  of  colonel.  The  composite  score  of  the  last  colonel  on  this 
list  is  66.4.  This  is  the  lowest  composite  score  any  job  may  have  and  still  be  identified  as  a  colonel's 
position.  A  job  with  a  composite  score  of  66.3  is  declared  to  be  the  first  or  most  highly  experienced 
lieutenant  colonel  position.  The  Policy  Board  identified  298  lieutenant  colonel  jobs,  (hunting  down  again 
m  the  same  fashion  as  was  done  for  the  colonel  jobs,  the  298th  job  has  a  score  of  50.6.  This  is  the  lowest 
score  any  job  can  have  and  still  be  declared  a  lieutenant  colonel  job.  Therefore,  scores  betu^een  the  values 
of  66.4  and  50.5  are  converted  to  the  grade  of  lieutenant  colonel.  The  remaining  grade  cutoff  scores  were 
determined  for  the  remaining  grades  in  a  similar  fashion.  Since  the  Policy  Board  judged  three  experience 
levels  within  each  grade  ( 16-point  scale),  except  for  general  officer,  this  information  was  used  to  construct 
a  table  of  cutoff  scores  which  equate  to  a  given  level  of  experience  within  each  grade.  After  the  initial  set 
of  cutoff  scores  had  been  determined,  each  cutoff  score  was  then  measured  from  each  other  score  above 
and  below  it  to  compute  the  inter-score  interval.  The  resulting  inter-score  intervals  (ISI)  with 
corresponding  cutoff  scores  and  grade  levels  are  presented  in  Table  5. 
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Table  5.  Inter-Score  Intervals  From  Initial  (Conversions 


Ifi- Point  .Scale. 

(•rttde,  and 

Hi  per ir  nee  Level 

Initial  Cutoff 

Si-ore  (.onvenmm 

Rraultin#  Inter- 
Score  Interval 

16  General 

15  High 

70.6  Si  above 

11  Middle  Colonel 

73.4 

6.2 

13  l.«w 

66.4 

7.(1 

4.6 

12  High 

61.8 

5.3 

1 1  Middle  It  Col 

56.5 

5.0 

10  l  ow 

50.6 

6.8 

0  High 

43.77 

7.3 

B  Middle  Major 

36.5 

5.7 

7  Low 

30.85 

4.0 

6  High 

26.0 

6.3 

5  Middle  Captain 

10.7 

6.7 

4  Low 

13.0 

4.3 

3  Lieutenant 

8.7  St  below 

As  indicated,  there  is  a  variation  between  inter-score  intervals  from  4.3  to  as  much  as  7.3  composite 
score  points.  In  the  1064  Policy  Board  sample  of  3.575  jobs,  the  16-point  conversion  table  which  resulted 
from  the  OCR  equation  distributed  ISls  between  cutoff  scores  in  a  relatively  equal  manner.  (Hazel  et  al  . 
I%6.  p.  17).  In  this  study  based  on  the  1.725  jobs  taken  from  the  3.575  job  set.  the  interval  sizes  are 
irregular.  As  the  number  of  positions  in  the  distribution  of  jobs  increases,  it  would  be  expected  that  ISI 
differences  would  decrease.  Since  the  conversion  table  was  to  be  applied  to  the  1 1,000  current  job  sample, 
irregular  interval  points  would  result  in  slight  distortions  of  the  number  of  jobs  allocated  to  any  given 
experience  level  or  grade.  To  avoid  the  occurrence  of  such  discrepancies,  the  ISI  values  were  fitted  with 
several  polvnominal  functions  to  smooth  out  the  variations  between  points.  The  best  fitting  curve  across 
the  ISIs  proved  to  be  a  second-degree  polvnominal  or  quadratic  function  of  the  general  form. 

V  =a  +  b,X  +  b2X2  (2) 

where  Y"  is  a  predicted  ISI  point  value  and  X  is  the  grade  and  experience  level .  ^  The  a  is  a  regression 
constant,  and  b|  and  b2  are  regression  weights  used  in  the  equation.  Based  on  these  computations,  minor 
adjustments  were  made  to  several  ISIs,  adding  or  subtracting  parts  of  composite  score  values  to  smooth  out 
the  variations.  The  ISI  adjustments  were  slight,  ranging  from  .07  to  1.2  score  points,  however,  the 
precision  attained  at  this  level  of  the  analysis  would  assure  less  distortion  when  the  conversion  table  would 
be  applied  on  a  much  larger  distribution.  The  resulting  conversion  table  should  allocate  a  more  even  or 
true-to-life  picture  since  in  the  real  world  there  are  over  62.000  jobs  in  the  non-aircrew  force  rather  than 
the  1.725  on  which  this  table  is  based.  The  research  conversion  table  is  presented  in  Table  6. 


'The  polynomial  form  of  a  second  -degree  rurvr  display*  one  c  hange  of  direction  rather  than  fitting  a  straight  line  to  the  It*- 
point  |S|  distribution  of  values  The  equation  resulted  in  a  curve,  slightly  booed  downward  in  the  middle  and  higher  on  the  ends 
running  across  the  grade  distribution  ISIs  from  lieutenant  to  colonel  The  slightly  bowed  or  depreaaed  portion  of  the  polvnomial 
curve  tilting  this  distribution  occurs  across  the  ISIs  at  the  grade  of  major  Smoothing  of  the  distribution  of  ISIs  permitted  the 
construction  of  a  more  uniform  conversion  table 


Table  6,  Kt'wirrh  Crade  (Conversion  Table 


Vriflbl^  < -ompoaitr 
(iumuUlivr  Srorr 

(dtnvrrlk  In 
tkperirnrr  l^vrl 

(.onvrrU 
to  (trade 

79.6  and  above 

15  Sr  Col 

( iolonel 

73.5  to  79.5 

14  Mid  Col 

67.4  to  73.4 

13  Jr  Col 

61.4  to  67.3 

12  Sr  Li  Col 

55.5  to  61.3 

II  Mid  Lt  fill 

Lt  (iolonrl 

49.6  to  55.4 

10  jr  Lt  (id 

43  7  to  49.5 

9  Sr  Maj 

37.7  to  43.6 

8  Mid  Maj 

Major 

31.7  to  37.6 

7  Jr  Maj 

25.6  to  31 .6 

6  Sr  (apt 

19.4  to  25.5 

5  Mid  (apt 

(aptain 

13.0  to  19.3 

4  Jr  (apt 

12.9  and  below 

3  Lt 

Lieutenant 

-  Thr  rututf  points  fur  eomposilr  irem  ffflrrlfd  shove  «rrr  developed  (mm  itir  l%4  \ ir  Kune  Collin 
Board  Thr  prrsrnl  tablr  dura  not  address  thr  problrru  of  grade  distributions  to  support  orderly  rarrrr  progression 
plana  Somr  adjustments  would  havr  to  br  madr  to  thr  eonvrrsion  tablr  ruloffa  to  accommodate  rarrrr  progression 
rrquirrmrnt*.  particularly  at  thr  tranaitton  Irvrla  from  captain  to  major  and  lirutrnant  to  captain 


Potential  Convention  Table  Modifications 

When  projections  were  made  using  the  (NCR  equation  and  the  research  conversion  table,  results 
indicated  a  requirement  for  more  majors  and  fewer  captains  and  lieutenants  than  called  for  b>  the  I  D). 
(Haael  A  Finstuen,  I960).  While  the  OCR-stated  requirements  for  major  positions  may  be  correct  in  terms 
of  the  content  and  responsibilities  associated  with  these  positions,  it  may  not  br  possible  for  the  Air  Force 
to  produce  this  many  majors  from  the  stated  caplain/lieutenant  base  because  of  present  career  progression 
plans.  Likewise,  for  captains  and  lieutenants,  the  conversion  table  reflected  a  requirement  for 
considerably  more  captains  than  lieutenants,  which  would  he  even  more  unreasonable  to  implement. 

The  problem  is  generated  by  the  fact  that  the  l%4  Policy  Board  recognised  very  few  non-aircrew 
positions  as  being  appropriately  filled  by  lieutenants;  yet  the  Air  Force  is  a  closed  system,  and  (except  for 
physicians)  captains  are  not  hired  directly  from  the  civilian  sector.  For  this  reason,  it  may  be  necessary  to 
identify  a  certain  proportion  of  the  lower-level  OCR  captain  positions  and  declare  them  to  he  lieutenant 
positions  on  the  IT>L.  Similarly,  it  may  be  necessary  to  declare  a  small  proportion  of  the  lower-level  OCR 
major  positions  to  be  captain  positions  on  the  I  III..  Such  actions  in  no  way  suggest  that  the  requirements 
ala  ted  hy  the  1964  Policy  Board  for  such  positions  are  incorrect;  rather,  it  is  a  recognition  of  the  need  to 
link  career  progression  programs  with  job  requirements  in  order  to  provide  reasonable  promotion  points 
It  should  be  recognized  that  the  OCR  equation  yields  a  continuous  distribution  of  composite  scores 
reflecting  the  job  responsibility  levels  for  positions.  This  would  permit  complete  flexibility  in  redefining 
rutting  points  to  incorporate  career  progression  considerations  into  an  operational  conversion  table  for 
lower  grades 
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Some  Final  Validity  Concerns  (N  =  1,725) 

Figure  .1  demonstrates  the  relationship  between  the  MET-judged  converted  composite  scores  using 
the  research  conversion  table,  and  the  Policy  Board  grades  assigned  to  these  1,725  officer  jobs.  On  the 
scale,  three  points  are  equivalent  to  one  officer  grade,  i.e..  points  7,  8,  and  9  constitute  the  rank  of  major. 
For  those  squares  on  the  diagonal,  there  is  exact  agreement  between  the  final  grade  equation  converted 
score  and  the  assigned  grade  given  to  the  same  job  by  the  Air  Force  Policy  Board.  Positions  plotted  one 
square  off  the  diagonal  constitute  a  judged  position  within  one-third  grade  level.  Positions  appearing  two 
squares  from  the  diagonal  square  likewise  constitute  a  MET  judged  position  within  two-thirds  grade  level 
in  agreement  with  the  Policy  Board.  Of  these  1.725  jobs.  1.614  or  94%  of  the  jobs  fall  on  the  diagonal  or 
within  two-thirds  grade  level  from  the  diagonal.  Of  the  remaining  jobs,  it  is  further  apparent  that  only  85 
or  2%  of  the  jobs  judged  by  METs  are  more  than  one  full  grade.  3  grade  levels,  away  from  the  diagonal. 
With  such  a  high  level  of  agreement  between  MET  members  and  the  Policy  Board,  it  appears  that  the 
newly  derived  grade  determination  equation  and  conversion  table  produce  very  dependable  results  and 
that  METs  can  routinely  duplicate  the  1964  Air  Force  Policy  Board  decisions  using  the  benchmark  scale 
procedures  and  factora. 


Converted  Composite  Score  from  Grade  Determination  Equation 


Figure  3.  Comparison  of  grades  assigned  to  1,725  positions  by 
the  1964  Policy  Board  and  the  present  OCR  Grade 
determination  equation. 


Comparisons  Using  N  =  1,725 


The  following  comparisons  may  be  made  between  the  Policy  Board  grade  allocations,  the  OCR 
converted  gTade  composite  scores,  the  HMD  grade  assigned  to  the  job,  and  the  actual  grade  of  the  job 
incumbent  for  the  1,725  case  sample  (Table  7). 


Table  7.  Comparisons  of  Grade  Level  for  1,725  Jobs 


UMD 

Incumbent's 

Grade  Level 

Policy  Board 

ogr 

Grade 

Actual  Grade* 

Colonel 

142 

133 

141 

128 

Lt  Colonel 

298 

332 

292 

311 

Major 

605 

541 

478 

419 

Captain/ 

Lieutenant 

608 

719 

814 

827 

Total 

1,725 

1.725 

1,725 

1,725 

^Includes  3*#  warrant  officers  and  one  incomplete  response. 

It  can  be  seen  that  the  OCR  equation  and  conversion  table  call  for  fewer  colonels  and  more  lieutenant 
colonels  than  either  the  Policy  Board  or  the  l  MD  authorization  for  these  1 ,725  jobs.  The  number  of  OCR 
grades  assigned  to  majors,  though  less  than  the  Policy  Board  grades,  is  more  than  the  number  of  grades 
allotted  by  the  UMD  or  the  actual  number  of  incumbents  in  the  grade.  This  is  the  same  general  finding  as 
from  previous  studies  which  have  stated  that  Air  Force  jobs  presently  filled  by  captains  and  lieutenants 
could  more  appropriately  be  filled  by  majors  based  on  the  content  of  the  job  and  the  level  of 
responsibility.  For  operational  use,  simple  adjustments  could  be  made  to  the  conversion  table  to 
encompass  the  present  career  progression  plan.  In  this  respect,  captains  and  lieutenants,  as  junior  officers, 
could  be  viewed  as  career  trainees  when  they  are  filling  jobs  requiring  a  higher  grade  level. 

The  correspondence  of  the  Policy  Board  decisions  to  the  OCR  composite  in  terms  of  the  appropriate 
grade  for  the  1,725  jobs  resulted  in  a  zero-order  correlation  coefficient  of  .90.  Table  8  also  reports  the 
correlations  between  various  other  grade  sources  for  the  1,725  jobs. 


Table  8.  Correlations  between  Policy,  OCR,  HMD,  and  Actual 
Grade  for  the  1,725  Criterion  Case  Sample 


OGR 

UMD 

Actual  Grade 
of  Inrumbent 

Policy  Board  Mean  Grade 
Rating 

.90 

.76 

.60 

OCR 

.80 

.62 

UMD 

.65 

The  Policy  Board  grade  judgments  were  made  with  a  16-point  scale.  As  shown  by  the  table,  the  highest 
order  of  agreement  was  reflected  between  the  OCR  and  the  Policy  Board;  however,  all  correlations  appear 
to  be  substantial  and  implementation  of  the  OGR  should  prove  to  be  consistent  with  existing  grades  based 
on  this  information.  The  critical  issue  would  not  address  the  changes  in  total  grade  structure  as  much  as 
the  required  realignment  of  grades  for  jobs  within  career  fields.  This  was  found  to  be  supported  in  the 
earlier  studies  in  examining  the  Policy  Board  decisions,  and  also  proved  to  be  implicit  in  the  judgments  of 
the  MET  members  in  terms  of  job  content  and  responsibility  for  various  job  descriptions  used  in  the  major 
field  lest  of  the  technology. 
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Inter-Hater  Reliability  Analyses  (N  =  1,725  jobs) 


Table  9  presents  the  inter-rater  reliabilities  (R^)  eomputed  for  an  average  of  15.61  ratings  using  an 
intraelass  correlation  coefficient  (Lindquist,  I *#53) . 


Table  9.  Inter-Rater  Reliabilities  (Rj^)  of 
MET  Raters  for  1,725  Policy  Board  Jobs 


fcqutfliuu  Nariablr 

Rkk* 

Factor  2  -  Special  Training  and  Work  Experience 

.92 

Factor  5  -  Communication  Skills 

.92 

Factor  7  -  Judgment  and  Decision-Making 

.91 

Factor  8  -  Planning 

.92 

Factor  9  -  management 

.91 

Predicted  tirade  Composite  Score 

.98 

ak  -  ratings  per  job. 


As  indicated,  there  is  high  agreement  among  MET  raters  as  to  the  level  of  job  content  and  responsibility 
using  benchmark  scales  on  the  five  equation  factors.  The  R^  value  may  be  interpreted  as  follows. 

Say  a  mean  value  is  computed  for  each  of  the  1.725  jobs  from  ratings  given  by  the  MET  raters. 
Further,  another  set  of  ratings  are  collected  from  a  similar  set  of  MET  raters  for  each  of  the  same  1,725 
jobs.  .Again  a  mean  value  is  computed  for  each  of  the  same  1,725  jobs  from  ratings  given  by  the  new  set  of 
raters.  If  a  correlation  coefficient  were  computed  between  the  first  and  second  sets  of  1 ,725  job  mean 
ratings,  the  result  would  be  the  R^  value,  which  is  then  an  index  of  the  stability  of  the  job  ratings  given 
on  a  factor.  Equation  factors  using  mean  ratings  are  all  'll  or  greater. 


Inter-Rater  Reliability  Analyses  (N  =  11,000  Current  Jobs) 

Inter-rater  reliabilities  were  also  computed  for  the  current  set  of  jobs.  This  allowed  assessment  of  the 
reliability  of  measurement  for  somewhat  less  ratings  per  job  (6.95  versus  13.61  obtained  in  the  1.725  case 
sample).  Reliabilities  reported  for  the  current  job  sample  are  correspondingly  lower  based  on  the  average 
number  of  ratings  per  job  as  shown  in  Table  10.  but  still  show  a  high  degree  of  stability,  with  the  predicted 
composite  score  dropping  only  .01  from  the  previous  .98.  This  provides  a  high  assurance  that  the  mean 
values  entering  into  the  regression  equation  were  stable  and  provided  a  sound  basis  upon  which  the 
projections  could  be  made.  Expectations  are  also  high  that  the  technology  will  provide  like  results  when 
used  on  a  day-to-day  basis  in  future  applications  at  the  METs.  The  high  agreement  assures  that  all  MET 
raters,  many  of  whom  participated  in  this  study,  would  be  able  to  apply  this  technology  with  little  or  no 
difficulty  and  could  produce  comparable  results  as  shown  here. 


Table  10.  Inter-Rater  Reliabilities  (R^fc)  of 
MET  Raters  for  11,321  Current  Jobs 


Equation  Variable 

R  * 

kk 

Factor  2  -  Special  Training  and  Work  Experience 

.85 

Factor  5  -  Communication  Skills 

.85 

Factor  7  -  Judgment  and  Decision-Making 

.84 

Factor  8  -  Planning 

.87 

Factor  9  -  Management 

.87 

Predicted  Grade  Composite  Score 

.97 

*k  *  6.95  ratings  per  job. 


Inter-Rater  Reliability  Analyses  (Rater  Types) 

The  150  MET  raters  responding  to  background  questionnaire  items  consisted  of  591  enlisted,  184 
civilian,  and  175  officer  MET  members.  Individual  inter-rater  reliabilities  for  each  rater  type  were 
computed  for  jobs  with  two  or  more  raters.  In  implementation  of  the  technology  at  the  METs,  each  rater 
will  be  asked  to  rate  a  complete  set  of  jobs,  so  the  average  number  of  ratings  per  job  will  be  the  same  as  the 
number  of  raters  making  the  ratings.  The  inter-rater  reliabilities  of  composite  scores  reported  here  were 
transformed  to  reflect  reliabilities  employing  seven  raters  using  the  Spearman-Brown  prophecy  formula 
(Guilford  &  Fruchter,  1973).  Results  are  indicated  in  Table  11. 


Table  11.  Inter-Rater  Reliabilities  (Rj^)  of  Seven  MET 
Raters  for  the  1,725  and  11,321  Case  Samples 


Rater  Type 

Policy  Board  Jobs 

Current  Jobs 

"kk 

Jobs  Rated 

Jobs  Rated 

Enlisted 

.96 

1,714 

.96 

9,817 

Civilian 

.96 

900 

.97 

4,200 

Officer 

.97 

1,365 

.98 

5,062 

Total 

.96 

1,725 

.97 

11,321 

Appendix  F  indicates  that  some  METs  have  available  from  15  to  18  raters  for  making  grade 
judgments,  while  other  smaller  METs  have  only  5  or  7  raters  available.  In  an  operational  setting,  the 
number  of  raters  available  for  making  benchmark  judgments  will  affect  the  reliability  of  judgments  on 
those  factors.  From  results  shown  in  Appendix  I,  it  is  evident  that  the  reliability  of  the  job  evaluation 
factors  varies  depending  on  the  number  of  raters  making  ratings.  Based  on  this  information,  it  is 
recommended  that  at  the  very  minimum,  five  raters  can  be  used;  however,  as  the  results  show,  seven  or 
more  raters  will  enhance  the  reliability  of  judgments.  All  19  factors  are  included  since  all  10  were  used  in 
the  field  test;  however,  only  five  of  the  factors  are  presently  used  in  the  grade  determination  process. 
Other  factors  may  be  of  operational  use  to  the  manpower  community  for  purposes  or  requirements  other 
than  job  evaluation  for  grade  assignment. 

In  addition  to  the  number  of  raters  suggested  for  use  in  an  operational  setting,  the  type  of  rater 
(enlisted,  civilian,  or  officer)  may  have  a  bearing  on  the  reliability  of  job  evaluation  factor  judgments. 


Appendix  J  demonstrates  that  from  the  1974  study  to  the  197b  study  offieers  eonsistently  demonstrated  a 
higher  reliability  of  judgment  than  the  other  two  groups.  Civilians  did  slightly  better  than  enlisted  raters, 
however,  all  raters  demonstrated  a  high  level  of  reliability  in  their  judgments.  Where  reliability  is  a 
concern  for  manpower  estimates,  a  tradeoff  can  he  made  between  the  number  of  raters  and  the  types  of 
raters.  Obviously,  if  only  five  or  six  raters  can  be  used  in  a  job  evaluation  project,  it  would  be  best  in 
reliability  terms  to  use  officer  raters  for  this  task.  Likewise,  if  many  raters  are  to  be  used,  enlisted 
personnel  could  be  selected  with  no  appreciable  decrease  in  reliability  results. 


IV.  SUMMARY 

This  report  documents  the  method  employed  by  Hazel  and  Finstuen  (1980).  as  well  as  the  resulting 
projections.  METs  were  used  in  the  development  and  refinement  of  a  method  which  systematically  and 
reliably  assigns  the  appropriate  military  grade  to  a  job  based  on  the  job  content  and  level  of  responsibility 
of  the  job.  The  field  test  documented  here  is  a  continuation  of  the  pilot  study  initialed  in  1971. 

This  pha  se  of  the  OCR  research  involved  refining  the  grade  determination  regression  equation  in 
order  to  prepare  the  technology  for  full  operational  implementation.  This  involved  removing  one  of  the 
variables.  MET  judgment  of  grade,  from  the  multiple  linear  regression  model  and  recomputing  the 
predictive  efficiencv  (R  =  .90)  against  the  derisions  of  the  Air  Forre  Policy  Board. 

A  further  extension  of  the  research  involved  construction  of  a  grade  conversion  table  which  converts 
predicted  composite  scores  from  the  regression  analysis  into  military  grade  and  level  of  experience.  Gutoff 
scores  were  computed  and  modified  to  compensate  for  slight  irregularities  of  inter-score  intervals  in  the 
table. 

The  conversion  table,  when  applied,  produces  a  substantial  increase  in  the  statement  of  requirements 
for  major  positions  over  those  of  the  present  I’DL.  This  is  consistent  with  the  findings  reported  in  the  1 96 1 
OGR  study.  However,  in  order  to  establish  a  reasonable  career  progression  system,  it  may  be  necessary  for 
a  number  of  borderline  major/captain  positions  to  be  listed  as  captain  on  the  l'I)L.  The  cutting  point  on 
the  OGR  conversion  table  separating  raptain  and  lieutenant  positions  should  be  established  by  Hq  1  SAP 
to  yield  the  most  appropriate  ratio  for  career  progression  purposes. 

Application  of  the  equation  and  the  conversion  table  resulted  in  highly  consistent  measures  and  were 
comparable  with  previous  decisions  made  by  the  Policy  Board. 

Inter-rater  reliabilities  for  sample  sets  were  computed,  varying  the  number  of  raters  and  ty  pes  of 
raters  based  on  the  Spearman-Brown  prophecy  formula. 

The  (X/R  technology  provides  a  scientifically  based  job  evaluation  system  for  established  non¬ 
aircrew  officer  positions  in  the  Air  Force.  Grade  evaluations  may  be  made  by  trained  METs  in  the  course 
of  their  manpower  surveys.  The  technology  also  provides  a  defensible  statement  of  Air  Force  grade 
requirements  based  on  the  content  and  responsibility  of  military  jobs.  The  methodology  presented  here 
may  serve  a  dual  purpose  in  that  the  (X7R  method  may  be  used  to  evaluate  a  single  job  and  that  grades  for 
jobs  in  the  non-aircrew  officer  force  can  be  compared  across  specialties  in  terms  of  content  and  job 
responsibility. 
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APPENDIX  A:  AIR  FORCE  OFFICER  POSITION  DESCRIPTION. 
INSTRUCTIONS  AND  PRIVACY  ACT  STATEMENT 


AIR  FORCE  OFFICER  POSITION  DESCRIPTION 


INSTRUCTIONS 

This  survey  is  directed  by  Hq  USAF  to  identity  and  describe  the  work  performed  by  officers  in  the  Air  Force. 
The  Air  Force  needs  precise  information  about  the  duties,  tasks,  and  requirements  of  officer  jobs  in  order  to  maintain 
the  classification  structure,  to  make  appropriate  grade  allocations,  to  define  incumbent  qualifications,  and  to  guide 
other  manpower  jnd  personnel  actions.  Participation  in  this  survey  gives  you  an  opportunity  to  provide  accurate 
information  about  your  |ob  in  support  of  improved  Air  Force  management. 

You  are  requested  to  complete  the  survey  according  to  the  following  instructions 

1.  ASSIGNMENT  INFORMATION  (Page  4)  Fill  in  the  required  data  or  check  the  one  box  in  each  block  that 
applies  to  you. 

2.  JOB  DESCRIPTION  (Pages  2  and  3)  On  these  pages  provide  typewritten*  information  which  accurately 
and  comprehensively  describes  your  job. 

a.  In  the  JOB  NAME  OR  TITLE  block  ,  record  a  name  or  title  which  is  descriptive  of  your  job. 

b.  In  the  JOB  CONTE  XT  block,  locate  your  job  within  the  organizational  structure. 

Examples  (1)  THIS  JOB  IS  IN  THE  HEAVY  EQUIPMENT  BRANCH  DIRECTLY  UNDER 
THE  BASE  MOTOR  POOL  COMMANDER,  WHO  REPORTS  TO  THE  M  &  S 
GROUP  COMMANDER. 

12)  THIS  JOB  IS  IN  THE  TARGETS  SECTION  OF  THE  OPERATIONS 
PLANNING  BRANCH  OF  WING  HQ. 

c.  In  the  blocks  under  DUTIES  AND  TASKS,  list  statements  that  describe  your  |ob.  Consider  significant 
work  activities  such  as  those  involved  in  commanding,  planning,  organizing,  directing,  monitoring, 
coordinating,  reviewing,  inspecting,  evaluating,  supervising,  and  operating.  Use  as  many  blocks  as  you 
consider  necessary.  The  statements  you  provide  should  clearly  define  your  |ob. 

Example  Duty  A.  DIRECTING  MATERIEL  CONTROL  FUNCTIONS 
TASKS  (1)  ASSIGN  PRIORITIES  TO  REQUISITIONS 

(2)  COORDINATE  REQUIREMENTS  FOR  MOBILITY  DEPLOYMENT 

(3)  MONITOR  SUPPLY  BUDGET 

(41  PROCESS  REQUESTS  FOR  LOCAL  MANUFACTURE  OF  ITEMS 
(5)  REQUISITION  TIME  CHANGE  ITEMS 

First,  list  all  the  major  duties  you  perform;  then  go  back  and  list  the  appropriate  tasks  under  each 
duty.  Describe  your  normal  |ob.  Omit  temporary  variations  in  your  work  which  are  not  part  of  your 
regular  assignment.  Ignore  additional  duties  unless  they  constitute  a  significant  part  of  your  job. 

d.  In  the  JOB  REQUIREMENTS  block,  enter  additional  statements  that  describe  any  unusual 
requirements  of  your  job  for  the  factors  below. 

COMMUNICATION  SKILLS  ORIGINALITY.  INGENUITY,  &  CREATIVENESS 
INTERPERSONAL  SKILLS  SPECIAL  TRAINING  &  WORK  EXPERIENCE 
WORKING  CONDITIONS  JUDGMENT  &  DECISION  MAKING 

FORMAL  EDUCATION 

Example*  (II  WORKING  CONDITIONS:  JOB  REQUIRES  APPROXIMATELY 
120  DAYS  TOY  ANNUALLY. 

(2)  SPECIAL  TRAINING  &  WORK  EXPERIENCE:  JOB  REQUIRES  A  30-DAY  AF 
COURSE  IN  SPECIAL  WEAPONS  DELIVERY. 

e.  In  the  JOB  SUMMARY  Block,  write  a  three-or  four-sentence  summary  description  of  your  job. 

f.  After  you  have  completed  pages  2.  3,  and  4,  sign  in  the  space  provided  on  page  4  and  submit  this 
form  to  your  supervisor. 

NOTE:  Supervisor  will  review  all  entries,  check  a  box  to  indicate  his  judgment  of  the  most  appropriate  grade  level  for 
this  job,  sign  the  form,  and  return  to  vour  Management  Engineering  Organization. 

*lf  typing  jervice  i*  not  available,  information  Jhould  be  dearly  printed  by  hand. 
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JOB  REQUIREMENTS 


JOB  SUMMARY 


AP  OFFICER  POSITION  DESCRIPTION 


JOB  DESCRIPTION  NUMBER  TO  BE  ASSIGNED  BY  MET 


ORGANIZATION  (10-44) 

SOCIAL  SECURITY  ACCOUNT  NUMBER 

1  [ 

Number  f 

V.  10-  19) 

Suffix 

(20-21) 

UDL  AUTHORIZED 
(MADE  FOR  YOUR 
POSITION 

YOUR  PRESENT 
GRADE 

GRADE  OF  YOUR 
IMMEDIATE 
SUPERVISOR 

Col 

(28) 

06 

Col 

129) 

06 

Civ 

(30) 

0A 

UC 

05 

L/C 

05 

Gan 

07 

Mei 

04 

Mai 

04 

CO 

o* 

Capt 

03 

Capt 

03 

uc 

05 

Lt 

02 

1/Lt 

Q* 

MN 

04 

W/O 

□  O 

2/Lt 

01 

Capt 

03 

W/O 

□  O 

1/Lt 

□  2 

2/Lt 

01 

BASE  OR  INSTALLATION  (45  78) 


YOUR  PRIMARY  AFSC 


MAJOR  AIR 
COMMAND 


Number 

123—26) 


TOTAL  MONTHS  IN 
YOUR  DUTY  AFSC 


TOTAL  MONTHS 
ACTIVE  FEDERAL 
MILITARY  SERVICE 


SEX 

(chock  one) 


□  □ 


LEVEL  OF  YOUR 
ORGANIZATION 

(chock  one) 

(45) 

DOO,  Hq  USAF  □  9 


Hq  Mai  AirComd  08 

Numbered  AF  r  i 

(or  Equivl  LJ  7 

Air  DMrioe  r— i  g 

lor  Equivl  U 

Wing  (or  Equivl  0  S 

Group  (or  Equivl  Q  4 

(or  Equivl  0  3 

Detaebment  i— i 

(or  Equlv)  l—l 2 


LEVEL  OF  YOUR 
JOB  WITHIN 
YOUR  ORGANIZATION 
(chock  one) 

_ _  .  (46) 

Commend  n 

Element  I—)  7 

Directorate,  Dept,  i — i  K 

Office  (or  Equivl  — ' 

Division  ,-,5 

(or  Equivl  I — I 


HIGHEST  LEVEL 
OF  YOUR 
EDUCATION 

(check  only  one) 
(47) 

Doctors)  Degree  r~| 


(or  Equivl 

Section 
(or  Equivl 

Unit  (or  Equivl 


Doctors)  Degree  ("[9 

Muster's  Degree  08 

Some  Post-  1 — 1 

Gred  Work  LJ7 

Bee  he)  or' t  Degree  06 

3  Years  College  05 

2  Yeers  College  04 

Y  Year  Collage  03 

H*h  School  Grad  02 

Non  High 

School  Grad  I — I ' 


3  Years  College 
2  Yeers  Collage 
1  Year  Collage 


M  F 

1  2  (43) 


YOUR  MAJOR  FIELD 
OF  STUDY 

(check  only  one) 
(48) 

Other  (Specify)  0  9 

Military  Science  0  8 

Social  Science  0  7 

Medical,  Legal  0  6 

Business,  . — , 

Management  I _ 1  5 

Engineering  0  4 

Arts,  Humardtres  0  3 

Physical  Science  0  2 

Education  0  1 


COMPLETED  BY: 

Signature 

Duty  Phona 

Date 

REVIEWED  BY: 

SUPERVISOR  CHECK  OSE -*■ 

Signature  of  Immediate  Supervisor 

OAFSC 

Data 

SUPERVISOR'S 

JUDGMENT 

OF  MOST 
APPROPRIATE 
GRADE  FOR 
THIS  JOB 

Gen 

(49) 

□  7 

Coi 

0  6 

L/C 

□  5 

Mq 

□  • 

Capt 

□  3 

Lt 

□  * 

Pqe4 
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AF  OFFICER  POSITION  DESCRIPTION 


Instructions  to  Incumbent 


To:  Designated  Officer 

1.  As  requested  by  Hq  USAF  Directorates  of  Manpower  and  Organization 
(PRM)  and  Personnel  Plans  (DPX) ,  the  Air  Force  Human  Resources  Laboratory 
has  undertaken  the  development  and  testing  of  a  technology  for  systemati¬ 
cally  evaluating  officer  positions  for  appropriate  grades  based  on  job 
descriptions  from  a  representative  sample  of  officers.  You  have  been 
selected  to  assist  in  fulfilling  this  requirement. 

2.  The  collection  of  job  information  is  being  accomplished  by  manpower 
and  management  engineering  personnel.  You  participation  in  this  project 
will  assist  the  Air  Force  in  achieving  reasonable  and  defensible  statements 
of  officer  grade  requirements. 

3.  Provide  all  requested  information  on  the  attached  Air  Force  Officer 
Position  Description  form,  then  give  it  to  your  immediate  supervisor  for 
review  and  rating  concerning  an  appropriate  grade  authorization.  If  your 
immediate  supervisor  is  not  available,  his  replacement  or  your  secondline 
supervisor  should  accomplish  the  requested  actions.  The  completed  form 
should  be  returned  to  your  Management  Engineering  Team  (MET)  or  manpower 
organization  within  10  working  days  after  your  receipt. 

4.  Since  your  name  and  SSAN  are  required  on  the  AF  Officer  Position 
Description,  a  Privacy  Act  Statement  is  provided  for  your  retention. 

5.  Accomplishment  of  the  Position  Description  is  authorized  under  Hq 
USAF/PRMRE  letter,  Subj :  Field  Test  for  Developing  Officer  Grade  Deter¬ 
mination  Procedures,  31  Oct  1975.  Any  questions  concerning  this  request 
should  be  referred  to  your  MET  commander  or  Manpower/Management  Engineering 
Team  project  officer. 


ATTACHMENTS 

1.  AF  Officer  Position  Description 

2.  Privacy  Act  Statement 
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>.  Authority 

10  U.S.C.  8012  Secretary  of  Air  Force,  Powers,  Duties,  De leg at  ion 
E09397,  22  Nov  41,  Numbering  System  for  Federal  Accounts  Relating 
Individual  Persons. 

bv  Compensation. 

To 

- J 

t  PRINCIPAL  FURFOSE(S) 

Tliis  Information  will  be  used  for  Air  Force  manpower  research  and  development;  to 
test  an  officer  grade  determination  technology,  and  guide  manpower  planning.  Use 
of  name  and  Social  Security  Account  Number  is  necessary  for  identification  of 
individual  records  and  job  or  position  numbers. 


1.  ROUTINE  USES 

Information  provided  by  individual  respondents  will  be  treated  confidentially  for 
the  stated  purposes.  Individual  identity  will  not  be  revealed  except  in  terms  of 
a  job  or  position  number.  Job  information  from  groups  cf  respondents,  who  will 
not  be  identified  by  name  or  Social  Security  Account  Number,  will  be  used  for  the 
following  purposes: 

a.  Evaluation  of  officer  grade  structures. 

b.  Construction  of  a  conversion  table  for  officer  grade  evaluation  procedures. 

c.  Manpower  and  personnel  research. 

d.  Other  manpower  management  systems  applications. 


4.  whether  disclosure  is  manoatorv  or  voluntary  and  effect  on  individual  OF  NOT  PROVIDING  INFORMATION 

Completion  of  the  Air  Force  Officer  Position  Description  by  job  incumbents  is  mandatory. 
Failure  to  provide  information  requested  will  significantly  detract  from  the  Air 
Force's  ability  to  evaluate  officer  grade  structures;  perform  manpower  requirements 
research;  and  perform  manpower  management  systems  applications. 


FORM  NUMMA  AMO  DATE 

AFPT  80-000-23,  12  Sep  75 

PRIVACY  ACT  STATEMENT 
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Field  Test  of  Officer  Grade  Determination  Procedures 


RATER  INSTRUCT!  CNS 


Tliis  research  study  is  designed  to  test  a  methodology  by  which  management 
engineering  and  manpower  personnel  can  apply  job  evaluation  factors  to 
objectively  determine  officer  grades  based  on  job  content  and  responsibilities. 
The  success  of  this  study  depends  on  your  cooperation  in  explicitly  following 
these  instructions. 

Your  MLT/Manpower  organization  commander  or  project  officer  is  responsible 
for  this  study  and  will  answer  any  questions  you  rray  have  about  requirements. 
Your  task  is  to  (a)  rate  a  number  of  officer  job  descriptions  on  10  job  evalu¬ 
ation  factors,  and  (b)  estimate  the  appropriate  grades  for  these  jobs  according 
to  a  grade  code  scale  provided.  Your  independent  judgments  on  the  evaluation 
factors  and  proper  grade  code  for  each  job  are  required,  so  do  not  confer  with 
other  raters  about  the  ratings. 

You  will  rate  one  car  more  folders  of  job  descriptions  as  assigned  by  your 
cxrmunder  or  project  officer.  For  this  purpose,  in  addition  to  these  instruc¬ 
tions,  you  will  be  provided  with  a  Rater  Form,  one  or  more  folders  of  job 
descriptions,  a  Job  Rating  Form  for  each  folder,  a  set  of  10  Job  Factor 
Scales,  and  a  Grade  Code  Scale.  Read  these  materials  and  eccanplish  the 
following  steps. 

Step  1.  Complete  the  Rater  Form.  Print  all  information  legibly  in  the 
spaces  prov idea. 

Step  2.  Examine  tlie  first  folder  of  job  descriptions  assigned  by  your 
;  reject  officer.  Each  folder  contains  approx  irately  25  descriptions  collected 
by  your  organization.  Do  not  change  the  sequence  of  the  descriptions  ir 
the  folder.  (Observe  that  a  job  number  has  been  entered  at  the  top  of  page  2 
of  each  description. 

A  separate  Jon  Hating  Form  has  bee’"  urepared  for  use  with  each  folder. 

Match  tne  ipproprrute  Job  Rating  Form  with  the  folder  under  examination  by 
verifying  that  the  job  numbers  listed  on  the  Job  Rating  Form  correspond  with 
the  jfb  numbers  in  the  folder.  Write  your  name  and  grade  at  the  top  of  the 
lob  Rating  Form. 

Stop  3.  Review  the  10  Job  Factor  Scales  on  pages  1  though  10  following 
these  instructions.  Notice  that  each  factor  has  9  levels  which  are  defined 
by  representative  job  titles.  Your  task  is  to  read  the  job  descriptions  in 
tne  folder  anu  then  rate  the  jobs  on  each  of  the  10  factors.  For  exanple, 
after  reading  the  first  job  description,  you  will  rate  that  description  frem 
1  to  9  on  Factor  1,  Formal  Education.  If  you  feel  the  job  description 
requires  about  the  same  level  of  education  as  the  job  titles  listed  at  level  4 
of  the  fomai  education  factor,  place  the  number  4  on  the  Job  Rating  Form 
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beside  that  job  number  under  Factor  1.  Then,  rate  that  job  on  all  the  other 
factors  (2  through  10)  following  the  same  prooedure.  In  each  instance, 
study  the  factor  definition  and  job  levels  before  naking  your  rating. 

Step  4.  Next,  examine  the  Grade  Code  Scale  on  page  11  after  these 
instructions.  Notice  in  the  Grade  Code  Scale  that  there  are  three  numbers 
associated  with  each  grade  level  except  General.  The  three  numbers  in  a 
set  are  used  to  shew  three  levels  of  experience.  For  instance,  a  4  represents 
a  Captain  with  a  short  time  in  grade,  a  5  represents  a  Captain  with  an  average 
time  in  grade,  and  a  6  represents  a  Captain  with  a  long  time  in  grade. 

Step  5.  After  understanding  the  Job  Factor  and  Grade  Code  Scales 
(steps  3  and  4) ,  start  making  your  ratings.  Real  the  first  job  description 
and  rate  it  on  all  10  factors  and  the  grade  code.  Write  tie  factor  ratings 
and  proper  number  from  the  Grade  Code  Scale  (1  to  16)  in  the  appropriate 
blocks  on  the  Job  Rating  Form  beside  the  job  number.  Next,  rate  the 
second  job  description  and  continue  until  all  the  descriptions  in  the  first 
folder  assigned  have  been  rated.  Repeat  the  rating  process  for  all  remaining 
folders  of  job  descriptions  collected  by  your  organization,  as  assigned  by 
your  project  officer. 

Step  6.  In  addition  to  the  folders  of  descriptions  collected  by  your 
organization,  rate  the  Job  Descriptions  from  a  Previous  Study  { approximately 
25  descriptions) .  *  Copies  of  these  descriptions  have  all  been  fastened  into 
a  separate  folder,  and  a  few  of  these  descriptions  may  consist  of  three 
pages.  If  any  of  the  job  descriptions  you  are  rating  appear  to  include 
outdated  equipment  or  organizational  level,  proceed  as  if  the  description 
were  current. 

Step  7.  When  you  have  finished  rating  each  of  the  jobs  in  all  of  the 
folders  assigned,  attach  each  completed  Job  Rating  Form  to  your  Rater  Form. 
Return  all  materials  to  your  project  officer  or  MET  oemnander  for  review 
and  consolidation. 


*  Certain  METs  designated  by  Hq  USAF/PFMRE  will  not  be  required  to  provide 
ratings  on  these  descriptions.  Project  officers  will  advise  raters  as 
appropriate. 
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APPENDIX  C:  BENCHMARK  FACTOR  SCALKS 


FACTOR  1  FORMAL  EDUCATION  I  he  ainoiiiil  ol  formal  education  requued  by  the  jot)  Consuls 
tin  .  i [ : '  . . .  nia  it  hi  In;:) i  m  bool .  college  university  .  oi  professional  school. 

LEVEL  9 

Si  ill  I  '  imI  OllVer,  Military  Allairs,  llq  USAE 

(  .  ii.  i  S\ stems  Design  Engineer 'Mathematician,  Hq  Major  Air  Command 
t  li  1 1- 1 .  II. ill  is  1 1>  riogiam.  Aerospace  lest  Wg 

Ltvt  l  a 

I’h ,  i-,  ist ,  ( icii'  ia'..  An  Force  Special  Weapons  Center 
Biocnwn  iimenial  Engineer,  Environmental  Health  laboratory 
Assisiani  I’roe.iam  Manager,  C-5,  Hi]  Major  Air  Command 

LEVtl  / 

Depots  Commander  tor  Maintenance,  Strategic  Missile  Wg 

Elect i ■  •  i i i c  Systems  Installation  Officer.  Electronics  Installation  Sq 

(  hu  t .  Mumtirms  Div.  llq  Numhcied  Air  force 

LEVEl  6 

t  1  a  i  i  t.it.i  Systems  A  Statistics.  O  mibat  Support  Gp 
i  )  •.••in  ,  i.  •  .mmandet .  Combat  Support  Gp 
A .  ,t  Slat  i  Electronics  Officer.  Combat  Evaluation  Gp 

LEVEL  5 

Assistant  DCS/l’crsonnel.  llq  Air  Weather  Service 

Aerial  Reconnaissance  Weather  Officer,  Weather  Reconnaissance  Sq 

Aiin.iiuent  Staff  Officer.  Inspector  General  Gp 

I  F.  V  E  I  4 

t  .'!<  A o -i.iinc.-rrf  A  l  ie  troun  s  Branch,  Consolidated  Aircraft  Maintenance  Sq 
(  In  I  I’ersonnel  Div,  Hi)  Combat  Support  Gp 
I  lei  i  ionic  Warfare  Officer.  Strategic'  Reconnaissance  Sq 

LEVEL  3 

Cl.iel  I’hotographic  Services  Branch,  Aeiospaee  Reconnaissance  Technical  We 
Cluel  Mu  in  lions  Maintenance  Branch.  Munition  Maintenance  Sq 
Ore.uu/alional  Maint  Office  Interceptor  Figlilei  Sq 

LEVEL  2 

duel  I  i.insportation  Trallii  M  m.ii'enienl .  Transportation  Sq 
Gump  Supply  Officer.  Aeiomcdicjl  Evacuation  Gp 
Base  Eueh  Officer,  l  ighter  Wg 

LEVEL  1 

\'<  hn  le  Maintenance  Officer.  Transportation  Sq 
Food  Service  Officer.  Combal  Support  Gp 
Cloihim’  Sales  Officer,  Combat  Support  Gp 

ft 
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FACYOR  2.  SPECIAL  TRAINING  AND  WORK  EXPERIENCE:  The  extent  to  which  the  job  requires 
knowledges  anil  skills  which  must  be  acquired  through  special  training  courses  or  on-the-job  experience 
Disregard  general  courses  given  by  Squadron  Officer  School,  Command  and  Staff  College,  or  War  College 

LEVEL  9 

Chic! .  Conti  act  Pricing  Branch,  Hq  USAF 

Chic i .  Military  Justice  Division,  Air  Div 

Space  Vehicle  Research  Officer,  Hq  AF  Special  Weapons  Center 

LEVEL  8 

Dirccior.  Reconnaissance  &  Electronic  Warfare  Operations,  Major  Air  Command 
Minuteman  Trajectory  Engineer,  Aerospace  Reconnaissance  Technical  Wg 
Chief,  Misstle/Nuclear  Safety  Division,  Technical  Training  Center 

LEVEL  7 

Missile  Combat  Crew  Commander,  Strategic  Missile  Sq 
Chief,  l.irget  Intelligence  Branch,  Strategic  Reconnaissance  Wg 
Chief.  Maintenance  Operations  Div,  Aerospace  Test  Gp 

LEVEL  6 

Chief.  Consolidated  Base  Personnel  Office,  Bomb  Wg 

Flying  Safety  Officer,  Hq  Tactical  Fighter  Wg 

Reconnaissance  Aircraft  Commander,  Strategic  Reconnaissance  Sq 

LEVEL  5 

Co-pilot  B-52,  Bomb  Sq 
Pilot,  Military  Airlift  Sq 

K.ular  (  valuation  Officer,  Hq  Major  Air  Command  (Overseas) 

LEVEL  4 

Chief,  Audio-Visual  Center,  Numbered  Air  Force 
Electronic  Warfare  Officer  B-52,  Bomb  Sq 
l  ilucatioii-Training  Officer.  Major  Atr  Command 

LEVEL  3 

Crypto  Operations  Officer,  Communications  Gp  (Overseas) 

Avionics  Officer,  Consolidated  Aircraft  Maintenance  Sq 
Flight  I  me  Maintenance  Officer,  Organizational  Maintenance  Sq 

LEVEL  2 

Photographic  Equipment  Maintenance  Officer,  Avionics  Maintenance  Sq 
Cluel,  Pay  &  Travel  Branch,  Combat  Support  Gp 
Photographic  Officer.  Technical  Reconnaissance  Sq 

LEVEL  1 

Base  Housing  Olficer,  Combat  Support  Gp 
Special  Service  Officer.  Fighter  Gp 
Transportation  Officer,  Instrumentation  Sq 


FACTOR  3:  WORKING  CONDITIONS:  I  lie  extent  to  which  the  job  involves  uncomlortabie  working 
coinliiioiis  Consider  such  conditions  as  isolation,  irregular  hours,  monotony,  prolonged  vigilance, 
extensive  1 DY,  and  pressure  to  meet  deadlines 


LEVEL  9 

I  oiwaid  An  Controller.  Tactical  Air  Support  Sq  (Overseas) 

Co  pilot  KC-135.  Air  Refueling  Sq 
Commander.  B-52,  Bomb  Wg 

LEVEL  8 

Pilot .  Reconnaissance,  Tactical  Reconnaissance  Sq  (Overseas) 
Tactical  Tighter  Pilot,  Tactical  Fighter  Sq 
Instiuctor  Pilot.  FB-1 1 1 ,  Bomb  Sq 

LEVEL  7 

Pilot.  Search  and  Rescue.  Aerospace  Rescue  &  Recovery  Sq 
Weapons  Controller,  Aircraft  Control  &  Warning  Sq 
Instructor  Navigator,  Transport.  Military  Airlift  Sq 

LEVFL b 

Flei  tn.  al  Engineer,  Site  Alteration  Task  Force 
Diieetoi  o|  Intelligence.  Numbered  Air  Force  (Overseas) 
Communications  Security  Officer,  Mobile  Communications  Gp 

LEVEL  5 

Weatliei  Forecaster.  Weather  Oct 

Chtel,  laigistics  Division,  Military  Airlift  Wg 

Nurse  Anesthetist.  Medical  Center 

LEVFL  4 

Transportation  Officer,  Transportation  Sq 

Director  of  Manpower  and  Organisation.  Air  Division 

Chtel  Airman  Personnel  Division.  Hq  Major  Air  Command 

LEVEL  3 

Asst  Staff  Judge  Advocate,  Combat  Support  Gp 

Clinical  Psychologist,  Medical  (.enter 

Chief  Military  Justice  Division,  AF  Missile  Test  Center 

LEVEL  2 

Special  Services  Officer.  Air  Base  Gp 
Ph.nmacy  Officer.  USAF  Hospital 
St  .ill  Chaplain.  Numbered  Air  Force 

LEVEL  1 

Ol  Titers'  Opc'n  Mess  Custodian.  An  Base  Op 
Custodian.  Non  Appropriated  Funds,  Art  Base  Gp 
Inhumation  Olficcr.  Bomb  Wg 


FACTOR  4  ORIGINALITY,  INGENUITY,  AND  CREATIVENESS:  The  extent  to  which  the  job 
tct|uii>  ■.  new  .nuJ  unique  methods,  approaches,  and  solutions  to  problems.  Consider  the  demand 
lor  novel  ideas  and  inventiveness. 

LEVEL  9 

lo  sean  h  Aviation  Physiologist,  USAF  School  of  Aviation  Medicine 
As  ionaiilaal  Engineer,  Propulsion,  Rocket  Propulsion  Lab 
I  lum. in  Perloimance  Engineer,  Electronic  Systems  Div 

LEVEL  8 

I  ovotiv  .Suit  Otlicer.  Hq  Air  Materiel  Area 

Manpowei  Management  Staff  Officer,  Hq  Major  Air  Command 

Dircci'u ,  Department  of  Aircraft  Maintenance  Training,  Tech  School 

LEVEL  7 

Missile  Safety  Officer,  Air  Force  Eastern  Test  Range 
Base  Deputy  Commander  for  Materiel,  Combat  Support  Gp 
Management  Engineering  Officer,  Hq  Air  Materiel  Area 

LEVEL  6 

Civil  fm  nicer.  Civil  Engineer  Sq 

Chief  Re  Entry  Vehicle  Maintenance  Branch,  Missile  Maintenance  Sq 
hiving  Safety  Officer,  Tactical  Fighter  Wg 

LEVEL  5 

Missile  Combat  Crew  Commander,  Strategic  Missile  Sq 
Reconnaissance  Pilot.  Tactical  Reconnaissance  Sq 
Commander,  Weather  Sq 

LEVEL  4 

Precision  Photographic  Services  Officer,  Strategic  Reconnaissance  Wg 
Clncl  l  ransportation  Traffic  Management,  Transportation  Sq 
Computer  Maintenance  Officer,  Hq  Major  Air  Command 

LEVEL  3 

Launch  Area  Maintenance  Officer,  Air  Defense  Missile  Sq 
Academic  Inslructoi-Undcrgraduutc  Pilot  Training,  Student  Sq 
Clinical  Laboratory  Officer,  USAF  Hospital 

LEVEL  2 

Asst  base  Equipment  Management  Officer,  Supply  Sq 
Co-Pilol  B-52,  Bomb  Sq 

Accounting  &  Finance  Officer,  Combat  Support  Gp 

LEVEL  1 

Asst  Medical  Supply  Officer.  Medical  Center 
Optometry  Officer.  USAF  Hospital 
Chief.  Administrative  Services,  Air  Base  Gp 
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FACTOR  5:  COMMUNICATION  SKILLS:  The  extent  to  which  the  job  requires  skill  in  oral  and 
written  communication.  Consider  the  complexity  and  variety  of  information  communicated  as  well 
as  the  level  of  the  individuals  and  agencies  involved. 

LEVEL  9 

Dire,  toi  of  Information,  Hq  Major  Air  Command 
Pol, iu  al  Military  Affairs  Officer,  Hq  USAF 
See  let  iry  of  the  Air  Staff,  Hq  USAF 

LEVEL  8 

Chiel  ol  Logistics  Division,  Hq  Numbered  Air  Force 
Asuouautical  Engineer,  Hq  Space  &  Missile  Systems  Org 
OSI  District  Commander,  Hq  District  OS1 

LEVEL  7 

Base  Civil  Engineer,  Air  Base  Gp 

Human  Performance  Engineer,  Electronic  Systems  Div 

Cornpi  roller,  Air  Base  Wg 

LEVEL  6 

Aviation  Physiologist,  Inspector  General  Gp 
Chemical  Engineer,  AF  Aero  Propulsion  Lab 
Administrative  Officer,  Electronics  Installation  Gp 

LEVEL  6 

Officci  Selection  Officer,  Det,  USAF  Recruiting  Gp 
Deputy  Commander,  Strategic  Missile  Sq 
Chief,  Sensors  Section,  AF  Special  Weapons  Center 

LEVEL  4 

Construction  Engineer,  Civil  Engineering  Sq  (Overseas) 

Squadron  Operations  Officer,  Combat  Crew  Training  Sq 
Accounting  4.  Finance  Officer,  Fighter  Gp 

LEVEL  3 

Commercial  Transportation  Officer,  Materiel  Sq 
Avionics  Officer,  Aircraft  Control  &  Warning  Wg 
Missile  Maintenance  Control  Officer,  Strategic  Missile  Sq 

LEVEL  2 

Fighter  Interceptor  Pilot,  Fighter  Intercetor  Sq 

Strategic  Missile  Complex  Maintenance  Officer,  Strategic  Missile  Sq 

Electronic  Warfare  Officer,  Tactical  Reconnaissance  Sq 

LEVEL  1 

Co-pilot.  Air  Refueling  Sq 
Navigator,  Bomb  Sq 

Helicopter  Pilot,  Aerospace  Rescue  &  Recovery  Sq 


FACTOR  6:  INTERPERSONAL  SKILLS:  The  extent  to  which  the  job  require;,  skill  in  dealing 
with  people  Consider  the  need  for  sensitiveness,  responsiveness,  persuasiveness,  sell -control, 
and  tact,  as  well  as  the  possible  consequences  when  such  skills  are  not  employed. 

LEVEL  9 

Staff  Chaplain,  Numbered  Air  Force 
Political  Military  Affairs  Officer,  Hq  USAF 
Commander,  Air  Refueling  Wg 

LEVEL  8 

Asst  Professor  of  Fconomics,  USAFA 

Academic  Instructor,  Dept  of  Chemistry  &  Physiology,  USAFA 
Security  Staff  Officer,  Hq  Major  Air  Command 

LEVEL  7 

Commissary  Officer,  Air  Base  Gp 
Wing  Director  of  Safety,  Bomb  Wg 
OSI  Detachment  Commander,  OSI  Det 

LEVEL  6 

Base  Procurement  Officer,  Flying  Training  Wg 
Instructor  Navigator  Bombardier,  Flying  Training  Wg 
Comptroller,  Air  Materiel  Area 

LEVELS 

Conimunications-Elcctronics  Staff  Officer,  Hq  Major  Air  Command 

Base  Supply  Officer,  Combat  Support  Gp 

Chief  Accounting  and  Finance  Division,  Combat  Support  Gp 

LEVEL  4 

Special  Services  Officer,  Services  Sq 

Cine  I,  Medical  Materiel  Services,  Medical  Center 

Maintenance  Supervisor,  Avionics  Maintenance  Sq 

LEVEL  3 

Air  Freight  Supervisor,  Aerial  Port  Sq 

Airborne  Electronics  Maintenance  Officer,  Fighter  Interceptor  Sq 
Construction  Engineer,  Civil  Engineering  Sq  (Overseas) 

LEVEL  2 

QIC  Photo  Laboratory,  Reconnaissance  Tech  Sq 
Avionics  Officer.  Avionics  Maintenance  Sq 

Precision  Photographic  Services  Officer,  Strategic  Reconnaissance  Wg 
LEVEL  1 

OK  Weapons  Services  Branch,  Munition  Maintenance  Sq 
Navigator,  Air  Refueling  Sq 
Co-pilot,  Air  Refueling  Sq 


44 


FACTOR  7:  JUDGMENT  AND  DECISION  MAKING:  The  importance  and  independence  ofjudgmcnis 
.in.l  ih .  I'.uuis  required  by  (lie  job.  Consider  (he  nature,  variety,  and  possible  impact  of  decisions. 

The  lesi  well  defined  the  guidance  for  decisions,  the  higher  should  be  the  rating;  while  the  more 
specific  and  detailed  the  guidance,  the  lower  should  be  the  rating. 

LEVEL  9 

Clue! ,  budget  Div,  Hq  Major  Air  Command 

Stall  legal  Officer.  Military  Affairs,  Hq  USAF 

Chief,  Weapon  System  Testing  Div,  Space  &  Missile  Systems  Org 

LEVEL  8 

Deputy  Commander,  Combat  Support  Gp 

Missile  Maintenance  Inspector,  IG,  Hq  Major  Air  Command 

LX  S/Comptroller,  Hq  Numbered  Air  Force  (Overseas) 

LEVEL  7 

Logistics  Officer,  Space  &  Missile  Systems  Org 

Experimental  Flight  Test  Officer,  Hq  Aeronautical  Systems  Division 

Cliief  of  Personnel,  Combat  Support  Gp 

LEVEL  t> 

Commander.  Organizational  Maintenance  Sq 

Missile  Safety  Officer,  Strategic  Missile  Wg 

Missile  Combat  Crew  Commander  (ICBM),  Strategic  Missile  Sq 

LEVEL  5 

Maintenance  Officer,  Strategic  Missile  Wg 
Base  Operations  Officer,  Combat  Support  Gp 
Aircraft  Commander  KC-135,  Air  Refueling  Sq 

LEVEL  4 

Munitions  Maintenance  Supervisor,  Munitions  Maintenance  Sq 
OIC,  Maintenance  Analysis  Branch,  Communications  Area 
Fighter  Interceptor  Pilot,  Fighter  Interceptor  Sq 

LEVEL  3 

Reconnaissance  Pilot,  Tactical  Reconnaissance  Sq 
Pilot.  Transport,  Military  Airlift  Sq 
Special  Services  Officer,  Services  Sq 

LEVEL  2 

Traffic  Management  Officer,  Transportation  Sq 
f  uels  Officer,  Air  Base  Gp 
Helicopter  Pilot.  Combat  Support  Gp 

LEVEL  1 

Recreation  Services  Officer,  Combat  Support  Gp 
Pharmacy  Officer,  USAF  Dispensary 
Photographic  Officer,  Reconnaissance  Teclfeica)  Wg 
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FACTOR  8.  PLANNING:  Lhe  extent  u>  which  planning  is  requited  by  the  job.  Consider  the  scope 
.ind  significance  . >r  woik  lor  winch  planning  is  done.  The  longer  the  time  span  Tor  which  planning  is 
done,  the  higher  the  rating  should  be. 

LEVEL  9 

Depot v  duel,  Plans  Division.  Ilq  Major  Air  Command 
Assl  Once  tor  ol  Wa:  Plan:,.  Ilq  Major  Air  Command 
Du'vloi,  Joint  Operations  Task  Force.  NORAD 

LEVEL  8 

duel ,  R  &  l)  Contracts  Dtv,  Alt  Force  Special  Weapons  Centet 
Management  Engineering  Officer,  Air  Materiel  Area 
Wing  Logistics  Officer,  Ah  Refueling  Wg 

LEVEL  7 

Maintenance  Control  Officer,  Bomb  Wg 
Dejnity  Commander.  Combat  Support  Gp 
Budget  Officer.  Air  Base  Gp 

LEVEL  6 

Operations  Otiicer.  Fighter  Interceptor  Sq 

Hospital  Administrator.  USAP  Hospital 

Chief.  Data  Services  Division.  Combat  Support  Gp 

LEVEL  5 

Chief  Career  Control  Branch,  Air  Base  Gp 
1  rallic  Management  Officer,  Transportation  Sq 
ProuucmciU  Officer,  Combat  Support  Gp 

LEVEL  d 

Mi, -ah  Combat  Crew  Commander,  Strategic  Missile  Sq 
Wing  Administration  Officer,  Military  Airlift  Wg 
Weapons  Officer,  Tactical  Fighter  Wg 

I  C  VEl.  3 

I  'ivtioi'j,  Warfare  Officer,  Bomb  Sq 

Medical  Administrative  Officer.  USAF  Dispensary 

Reconnaissance  Pilot,  Tactical  Recon  Sq 

LEVEL  2 

Fighlci  Interceptor  Pilot,  Fighter  Interceptor  Sq 

Regislrai.  Medical  Center 

Security  Police  Officer,  Security  Police  Sq 

LEVEL  1 

Plight  Nuisc.  Acromedical  Evacuation  Sq  (Overseas) 

Weather  Pore  astei ,  Weather  Del 
Optometrist.  Medical  Center 


.  ACIOH  S  M M  A  i  ■  L  M  l  N  I  i  !  i. ,  1  1 1|  t  \e.  ul  ivt  ml.  iiu.'t  . .  i!  ’ 
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LEVEL  9 

Dm'  !.  i  hi  Budget ,  llq  Ma|<ir  An  Command 
('■  kiiiii.iii.Iiv  .  1  mtut  Suppmi  Gp  (( )v  isc.is) 

W  ill;'  (  nnniuiuli'i  lactic al  Control  Wg  (Overseas) 

LEVEL  6 

W  i  .iC  <  I'lninamli  i .  Aerospace  Kescue  &  Keen  very  Wg 
Clue!  ol  Operations.  Sliatcg.it  Missile  Sq 
IVpuly  Commander.  An  Base  (■() 

LEVEL  7 

Maintenance  Sup*  rvisor.  Avionics  Maintenance  Sq 
Squadron  Opeiations  Olt'iccr,  Combat  Crew  Training  Sq 
Base  Accounting  &  Finance  Officer.  Flying  Training  Wg 

LEVEL  6 

(Tint  ,  v  i  ii-i'lnJatcJ  Base  Personnel  Office,  Combat  Support  Gp 

Base  Piocnreinent  Officer.  Pilot  Training  Wg 

Helicopter  Squadron  Operations  Officer.  Flying  Training  Sq 

LEVEL b 

1  raffle  Management  Office! .  Transportation  Sq 

Base  (  onimiinicaiions  Maintenance  Officer.  Conitminicalions  Sq  (Overseas) 
Missile  Combat  Crew  Commander.  Strategic  Missile  Sq 

LEVE1  d 

•  I  .  '  I  i  Hines  ( ipeiatioiis  Division.  Civil  Engineering  Sq 
t  luti  Plroio  I  valiiation  Branch,  Photographic  Sq 
Base  Fuels  (Miner.  Supply  Sq 

LEVEL  3 

Pnm.ir.  Pilot  I  mining  lusUuuot.  Pilot  Training  Sq 
Spate  Si.iveillanie  Olfitei.  Aerospace  Support  Sq 
An  7  ijlltc  (  ontroller.  Communication  Sq 

LEVEL  ? 

Adnumsttative  Olfiecr,  Ait  Base  Sq 
Data  Vi  vices  Officer,  Combat  Support  Gp 
l  n  tii  ,il  lighter  Pilot  .Tactical  Fighter  Sq 

LEVEL  1 


i  hint  il  Psychologist.  1  SAP  Hospital 
I'sy  t  In  U i rc  Social  Worker,  USAP  Hospital 
I Iciit  o|.ii  r  Pilot  Single  Kotor,  Air  Base  Sq 


FAC.  ON  10:  HISK  I  Ik  .■ Atenl  to  which  the  job  requires  exposure  to  risk  of  death  Of  severe 

miurv  it  immic  imu- 

LEVEL  9 

I-.,  w  iiil  An  Controller,  Tactical  An  Support  Sq(Overseas) 

la.  heal  hgi.ict  t*ilot.  r.n  heal  l  loiter  Sq  I 

lush  t  tor  Pilot.  lac  ueal  Fightei.  Combat  Crew  Training  Sq  \ 

i 

i 

l.EVEl  a 

Pilot  I  M  i.f>.  Tactical  Electronic  Warfare  Sq  (Overseas) 

Navii’.oo.  Tmop  Carrier,  Military  .Airlift  Sq 
Helicopter  Pilot,  Aerospace  Kescue  &  Recovery  Sq 

LEVEL  7 

Instructor  Missile  1  junch  Officer,  Strategic  Missile  Sq 
Arctic  Survival  Training  Officer,  Strategic  Wg  (Overseas) 

Chid .  Propel  ants  Programming  Br,  AF  Rocket  Propulsion  Laboratoiy 

LEVEL  6 

Commando;.  Radar  Sq 

Research  Biochemist,  School  of  Aerospace  Medicine 
Chemist.  Air  Force  Materials  Laboratory 

LEVEL  5 

Chief.  General  Investigations  Div,  Hq  District  0S1 
Base  Veterinarian,  USAF  Hospital 
Commander,  Civil  Engineering  Sq 

LEVEl  4 

Meeiutni;  d  Engineer,  Space  &  Missile  Test  Center 
Ait  Traffic  Control  Officer,  Communications  Sc 
Instructor,  Institute  for  Professional  Development 

LEV! L  3 

Medical  Suppiy  Officer.  USAF  Hospital 

Chief,  Engineering  Standards  Branch,  Communications  Region 

Chief  Machine  Processing.  Air  Base  Gp 

LEVEL  2 

Recreation  Services  Officer.  Combat  Support  Gp 
Cliapi. mi.  Combat  Support  Gp 

Manpower  Management  Staff  Officer,  Flying  Training  Wg 
LEVEL  1 

Custodian  Non  Appropriated  Funds.  Air  Base  Wg 
Clothing  Sales  Store  Officer,  Air  Base  Wg 
Instructor,  French,  Dept  of  Foreign  Languages,  USAFA 
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GRADE  CODE  SCALE 


GRADE 

CODE 

1 

LIEUTENANT 

2 

3 

4 

CAPTAIN 

5 

6 

7 

MAJOR 

8 

9 

10 

LIEUTENANT  COLONEL 

11 

12 

13 

COLONEL 

14 

13 

GENERAL 


16 


I/'/’/  \l)l\  I  PHOJM  TOFFM.KK  IYSTRI  CTlOYs 


SUBJECT: 

TO: 


Field  Test  of  Officer  Grade  Determination  Procedures 
MET/Manpower  Organization  Commander 

1.  The  Air  Force  Director  of  Manpower  and  Organization 
has  asked  the  Air  Force  Human  Resources  Laboratory  to 
assist  in  the  development  and  testing  of  operational 
grade  determination  procedures. 

2.  As  Commander  of  one  of  the  organizations  designated 
for  inclusion  in  this  study,  you  are  responsible  for 
insuring  prompt  compliance  with  the  requirements  speci¬ 
fied  in  Attachments  1  through  8.  All  necessary  materials 
are  enclosed. 

3.  Please  study  the  MET  Commander/Pro ject  Officer  Instruc¬ 
tions  and  review  the  package  of  materials  provided.  All 
the  specified  requirements  should  be  accomplished  and 
materials  returned  within  40  work  days  after  receipt. 


ATTACHMENTS 


1. 

MET  Cdr/Proj  Off  Instructions 

2. 

Notification  Form 

3. 

Job  Sample  Specifications 

4. 

SSAN  Roster 

5. 

Job  Incumbent  Ltr  w/Forms 

6. 

Job  Descriptions  from  Previous 

Study 

7. 

Rater  Instructions  w/scales  and 

forms 

8. 

Return  Labels  and  Envelopes 

SO 


Field  Test  of  Officer  Grade  Determination  Procedures 


MET  Commander /Pro j ect  Officer  Instructions 

Each  MET/Manpower  organization  commander  is  responsible  for  accurate 
accomplishment  of  the  steps  specified  in  Phases  1,  II,  and  III  below. 

This  may  be  done  either  by  the  MET  commander  or  a  project  officer.  Essen¬ 
tially  this  field  test  of  officer  grade  determination  procedures  involves 
MET/CBPO  identification  and  selection  of  officer  incumbents  to  complete 
job  descriptions,  MET  collection  of  job  descriptions,  and  MET  evaluation 
of  job  descriptions. 


Phase  I 

This  phase  will  require  close  coordination  between  METs  and  local 
or  servicing  CBPOs  to  identify  and  select  officer  job  incumbents  to 
complete  job  descriptions.  Maximum  CBPO  participation  may  be  obtained 
for  this  phase. 

Step  1.  According  to  Air  Force  records,  officers  in  the  DAFSCs  and 
grades  given  in  attachment  3  are  available  to  you  for  collection  of  job 
descriptions.  These  officers  are  identifiable  through  your  CBPO  or  CBPOs 
listed  in  attachment  3.  In  conjunction  with  the  CBPO,  identify  and  locate 
the  number  of  officers  specified  by  DAFSC  and  grade  in  attachment  3. 
Officers  selected  should  represent  a  variety  or  as  many  different  4-digit 
DAFSCs  (including  shreds)  as  available  within  each  2-digit  DAFSC  category. 
Officers  who  are  assigned  as  overages  should  not  be  selected. 

In  previous  officer  grade  evaluation  project,  91  CONUS  METs  partici¬ 
pated  in  the  development  of  a  grade  determination  technology.  For  these 
METs  only,  attachment  4  provides  a  SSAN  roster  of  individuals  who  accom¬ 
plished  job  descriptions  in  late  1974.  Do  not  include  these  individuals 
in  the  present  study  if  they  are  currently  performing  the  same  job. 

Phase  II 


Step  1.  After  the  officers  in  the  DAFSCs/grades  specified  in  attach¬ 
ment  3  have  been  selected,  contact  these  officers  to  complete  Air  Force 
Officer  Position  Descriptions.  Forward  each  officer  job  incumbent  a 
letter  of  instructions.  Privacy  Act  Statement,  and  Air  Force  Position 
Description  as  shown  in  attachment  5.  If  a  job  incumbent  you  have  selected 
is  not  available  due  to  transfer  or  extended  TDY ,  substitute  another  officer 
in  the  same  DAFSC  and  grade  for  completion  of  a  job  description. 


me  air  ro 


i.  V.  VI  ) 


Step  .  _ 

pleted  and  returned  to  you  within  10  working  davs  after  their  receipt  by 
en.ii  individual  officer  selected.  Contact  the  selected  officers  before 
first  week  to,  insure  they  received  the  job  description 

making  progress  towards  completion  of  Position  Descriptions, 


t he  end  of  the 
material  and  are 


Step  3 ■  When  the  AF  Officer  Position  Descriptions  have  been  returned 
to  your  organization/MET ,  carefully  inspect  all  pages  of  the  descriptions 
for  completeness  Review  pages  2  and  3  of  the  job  description  to  insure 
that  information  concerning  job  title,  job  context,  and  job  summary  is 
provided.  The  listing  of  duties  and  tasks  should  be  legible,  and  should 
appear  reasonably  consistent  with  other  information.  Screen  page  4  f-i>sign- 
ment  information),  insuring  that  all  blocks  are  completed,  particularly  the 
supervisor's  judgment  of  the  most  appropriate  grade  for  the  job.  If  this 
information  is  not  provided,  the  position  description  must  be  returned  for 
accomplishment.  If  the  immediate  supervisor  is  not  available  for  an  extended 
period,  his  replacement  or  the  second-line  supervisor  should  accomplish  this 
rating. 

Step  4.  Before  proceeding  with  Phase  III,  other  administrative  process¬ 
ing  of  the  job  descriptions  must  be  accomplished  by  METs.  A  job  number  iden¬ 
tifier  must  be  entered  in  the  blocks  at  the  top  of  page  4  of  each  position 
description.  These  job  numbers  are  necessary  for  future  reference  by  METs 
to  identify  which  jobs  were  surveyed  in  the  present  project  so  as  to  avoid 
duplication  or  resurvey  of  the  same  jobs  in  any  subsequent  application  of 
technology. 

Step  5.  For  all  completed  job  descriptions,  transcribe  each  job  number 
from  page  4  to  the  top  right  side  of  page  2  of  the  form.  Randomly,  sort 
the  completed  descriptions  into  approximately  equal  sets  of  25  descriptions. 
Fold  each  description  so  that  pages  2  and  3  are  on  the  outside  with  page  2 
on  top.  Punch  holes  on  the  left  margin  of  the  folded  descriptions  and  then 
fasten  each  of  the  sets  of  25  into  the  folders  provided.  Make  sure  that 
each  job  description  is  folded  and  fastened  so  that  only  pages  2  and  3  can 
be  read  by  raters. 

Step  6.  For  each  folder,  list  the  job  numbers  from  page  2  of  the  job 
description  on  a  Job  Rating  Form  such  that  there  is  one  Job  Rating  Form 
associated  with  each  set  of  approximately  25  descriptions.  This  is  a  cri¬ 
tical  step,  so  make  sure  the  job  numberes  are  entered  accurately  and  legibly 
(typing  optional) . 

A  Job  Rating  Form  with  job  numbers  listed  is  necessary  for  each  rater 
of  the  descriptions  in  each  folder.  The  Job  Rating  Forms  for  each  folder 
may  be  machine-duplicated.  If  such  services  are  not  available,  a  separate 
form  must  be  prepared  for  each  rater.  Sufficient  copies  of  the  Job  Rating 
Form  are  provided  for  this  purpose.  The  number  of  raters  and  Job  Rating 
Forms  required  are  outlined  in  Phase  III. 

Step  7.  In  addition  to  the  AF  Officer  Position  Descriptions  collected 
for  this  project,  certain  METs  will  rate  approximately  25  job  descriptions 
from  a  previous  study.  The  folder  containing  the  set  of  descriptions  for 
your  organization  is  given  in  attachment  6.*  Observe  that  each  of  these 


*  Certain  METs  designated  by  Hq  USAF/PRMRE  will  not  be  requested  to  accom¬ 
plish  ratings  on  this  folder  of  descriptions.  Attachment  6  listed  in  the 
MET  Commander’s  letter  has  been  withdrawn  if  appropriate.  However,  all 
other  requirements  outlined  in  these  instructions  are  applicable  to  such 
METs. 


descriptions  from  the  previous  study  has  already  been  assigned  a  six-digit 
job  number,  found  at  the  top  of  each  page.  Using  these  job  numbers,  pre¬ 
pare  a  separate  Job  Rating  Form  for  this  folder  of  descriptions  for  each 
rater  as  described  in  Step  6. 


Phase  III 

This  phase  requires  manpower  and  management  engineering  personnel  at 
your  location  to  rate  the  job  descriptions  in  accordance  with  the  Rater 
Instructions  (attachment  7).  Clerical  personnel  will  not  be  used  as  raters. 

In  METs  with  7  to  13  personnel,  obtain  ratings  from  a  minimum  of  the  7  most 
experienced  personnel  on  all  position  descriptions.  In  fewer  than  7  personnel 
are  available,  obtain  ratings  from  all  personnel.  For  larger  METs  with  14 
or  more  potential  raters,  divide  the  folders  of  position  descriptions  into 
approximately  equal  groups,  such  that  each  group  of  folders  is  rated  by  a 
minimum  of  7  raters.  MET/Manpower  commanders  and  project  officers  may  serve 
as  raters. 

Step  1.  Provide  each  rater  with  a  set  of  Rater  Instructions,  a  Rater 
Form,  folders  of  position  descriptions.  Job  Rating  Forms  with  job  numbers 
listed,  and  other  rating  material  shown  in  attachment  7.  Folders  of  posi¬ 
tion  descriptions  with  associated  Job  Rating  Forms  can  be  arranged  or  de¬ 
livered  to  raters  in  any  convenient  sequency,  except  that  the  one  folder 
containing  approximately  25  descriptions  from  a  previous  study  should  be 
rated  by  each  rater  after  some  experience  has  been  gained  with  the  other 
folders . 

Step  2.  Insure  that  all  raters  accomplish  the  requirements  outlined 
in  the  Rater  Instructions.  Each  rater  will  complete  a  Rater  Form,  which 
includes  the  name  and  SSAN.  If  Privacy  Act  questions  arise,  tell  raters 
that  name  and  SSAN  will  be  used  merely  to  collect  the  collate  data,  with 
individual  rater  identity  treated  only  in  aggregate  terms. 

Step  3.  On  receipt  of  materials  fr;m  raters,  as  project  officer  you 
should  review  all  Rater  Forms  and  Job  Rating  Forms  for  completeness,  and 
insure  that  the  appropriate  number  of  Job  Rating  Forms  is  attached.  Do 
not  change  ratings  assigned  by  raters. 

As  summary  of  required  actions,  a  checklist  is  provided  in  Appendix  B 
for  your  convenience.  Before  return  of  project  materials,  all  requirements 
outlined  in  the  checklist  should  be  accomplished. 

Step  4.  When  all  Rater  Forms  and  Job  Rating  Forms  have  been  thoroughly 
checked  as  prescribed,  assemble  and  forward  all  project  materials  to  the 
Occupational  and  Manpower  Research  Division,  AFHRL/ORE,  Stop  if 63,  Lackland 
AFB ,  TX  78236.  A  return  address  label  has  been  provided  (attachment  8). 


MET  Commander/ Project  Officer  Instructions 


Checklist 

1.  Review  the  entire  package  of  project  materials  provided  and  carefully 
read  the  MET  Commander/Proj ect  Officer  Instructions. 

2.  Return  the  completed  Notification  Form  to  AFHRL/ORE,  within  5  working 
days.  Provide  the  date  of  the  UPMR/MPAD  used  to  derive  job  numbers. 

3.  Through  MET/CBPO  coordination,  identify  and  locate  officers  in  the 
appropriate  DAFSC/grades  specified  for  your  MET/ organization .  Select  these 
officers  and  obtain  a  completed  AF  Officer  Position  Description, 

4.  Check  each  position  description.  Insure  completion  of  the  supervisor's 
judgment  of  the  appropriate  grade  for  each  job. 

5.  Determine  and  assign  a  job  number  identifier  to  each  AF  Officer  Position 
Description.  Accomplish  the  administrative  processing  required  before  commenc¬ 
ing  ratings  of  descriptions. 

6.  Identify  and  contact  the  prescribed  number  of  raters  outlined  in  Phase 
III.  Provide  the  necessary  material /forms  and  obtain  ratings  from  manpower 
and  management  engineering  raters  in  accordance  with  Rater  Instructions. 

7.  Check  all  Rater  Forms  and  Job  Rating  Forms  for  accuracy  and  completeness. 
Insure  that  complete  sets  of  ratings  have  been  provided  by  the  prescribed 
number  of  raters. 

8.  Assemble  all  forms  and  descriptions  and  return  all  study  materials 
to  AFHRL/ORE  not  later  than  12  Mar  1976. 


APPENDIX  F:  122  PARTICIPATING 
MANAGEMENT  ENGINEERING  TEAMS  (MAIN  STUDY  1976) 
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Management  Engineering  Teams  and  Raters 
Participating  in  the  Main  Study  (1976)  by  Central 
Base  Processing  Office  (CBPO)  (iode 


CBPO 

Code 

MKT 

UnatioM 

Number 
of  Raters 

CBPO 

Code 

me r 

Locution 

Numbers 
of  Raters 

1 

AF 

Howard 

6 

41 

FW 

F.E.  Warren 

7 

2 

AH 

Alconbury 

6 

42 

GB 

George 

7 

3 

AM 

Altus 

5 

43 

GM 

Grand  Forks 

9 

4 

AT 

Andersen 

7 

44 

GW 

Griffiss 

9 

5 

All 

Andrews 

8 

45 

HB 

Hahn 

8 

6 

AX 

Athenai  Apt 

7 

46 

HF 

Hancock 

6 

7 

AY 

Aviano 

7 

47 

HH 

Pentagon 

9 

8 

BB 

Barksdale 

11 

48a 

HI 

Hickam 

15 

9 

BO 

Beale 

7 

48a 

HI 

Hickam 

(15) 

LO 

BF 

Bentwaters 

7 

50 

HP 

Hill 

14 

II 

BH 

Bergstrom 

7 

51 

HS 

Holloman 

5 

12 

BL 

Bitburg 

7 

52 

HV 

Homestead 

5 

13 

BN 

Blvtheville 

7 

53 

KB 

Kadena 

8 

14 

BP 

Bolling 

5 

54 

KF 

Keesler 

11 

15 

BV 

Brooks 

7 

55 

KH 

Kelly 

14 

16 

BX 

Grissom 

8 

56 

KJ 

Kelly/ 

USAFSS 

11 

17 

CC 

New  Amster- 

dam 

7 

57 

KM 

Kincheloe 

7 

18 

CD 

Cannon 

7 

58 

KU 

Kunsan 

7 

19 

CF 

Carswell 

7 

59 

KV 

Kirtland 

7 

20 

CH 

Castle 

9 

60 

KY 

K.l.  Sawyer 

7 

21 

CK 

Chanute 

8 

61 

LA 

Lackland 

7 

22 

CL 

Charleston 

7 

62* 

LE 

l^angley 

21 

23 

CO 

Columbus 

6 

63a 

LE 

Langley 

(21) 

24 

CP 

Clark 

16 

64 

LJ 

Laughliu 

5 

25 

CZ 

Craig 

5 

65 

LK 

L.G.  Hanscom 

7 

26 

I)F 

Davis- 

Monthan 

7 

66 

I.P 

Little  Rock 

8 

27 

I)J 

AFAFC 

5 

67 

LQ 

Rickenbacker 

12 

28 

DM 

Dover 

7 

68 

LS 

Ixiring 

7 

29 

DT 

Duluth 

6 

69 

LI 

lyos  Angelus 

? 

30 

DW 

Dyess 

7 

70 

LW 

la>wrv 

7 

31 

EB 

Edwards 

8 

71 

LY 

Luke 

8 

32a 

ED 

Eglin 

16 

72 

MA 

MacDill 

5 

33a 

ED 

Eglin 

(16) 

73 

MB 

Malstrom 

7 

34 

EH 

E  i  e  1  s  o  n 

(included  w / 

EL) 

(8) 

74 

MI) 

March 

7 

35 

EJ 

Ellsworth 

7 

75 

ME 

Mather 

7 

36 

EL 

Elmendorf 

8 

76 

MG 

Maxwell 

16 

37 

EM 

England 

77 

MH 

McChord 

8 

38 

F,P 

Peterson  Fid 

17 

78 

MK 

Mcfionnell 

7 

39 

FC 

Fairchild 

7 

79 

ML 

Mildenhall 

7 

40 

FM 

Ft  Belvoir 

5 

80 

MN 

McGuire 

6 
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CBPO 

Code 

MET 

Location 

Number 
of  Raters 

CBPO 

Code 

MET 

Location 

Numbers 
of  Raters 

81 

MP 

Minot 

6 

104 

SP 

Shaw 

6 

82 

MT 

Moody 

7 

105 

SQ 

Sheppard 

14 

83* 

MU 

McClellan 

10 

106 

TE 

Tinker 

12 

84* 

MU 

McClellan 

(10) 

107 

TJ 

Torrejon 

7 

85 

MW 

Mt  Home 

6 

108 

TP 

Travis 

7 

86 

MY 

Myrtle  Beach 

5 

109 

TX 

Tyndall 

10 

87 

NJ 

Nellis 

7 

110 

UP 

Upper 

Heyford 

7 

88 

NV 

Norton 

10 

111 

US 

USAF 

Academy 

7 

89 

OD 

Offutt 

18 

112 

VQ 

Vandenberg 

8 

90 

OP 

Osan 

5 

113* 

WE 

Wright-Pat- 

terson 

30 

91 

PF 

Patrick 

7 

114* 

WE 

Wright-Pat- 

terson 

(30) 

92 

PJ 

Pease 

8 

115 

WG 

AF/OSI 

7 

93 

PS 

Piattsburg 

7 

116 

WM 

Webb 

5 

94 

PV 

Pope 

6 

117 

WT 

Whiteman 

7 

95 

RF 

Ramstein 

8 

118* 

WU 

Lindsey 

7 

96* 

RJ 

Randolph 

17 

119* 

Wl! 

Lindsey 

(7) 

97* 

RJ 

Randolph 

(17) 

120 

wv 

Williams 

6 

98 

RM 

Reese 

6 

121 

wz 

Wurtsmith 

7 

99 

RT 

Richards 

Gebaur 

14 

122 

YM 

Yokota 

8 

100* 

RX 

Robins 

7 

101* 

RX 

HQAFRES 

(Robins) 

(7) 

102 

SF 

Scott 

7 

103 

SM 

Seymout 

Johnson 

7 

Total 

932 

“indicates  two  METs  assigned  to  the  same  base  representing  more  than  one  major  command.  Raters  cooperated  and  are  counted 
only  once.  Consequently  there  were  932  individual  MET  members  who  participated  in  the  1976  study,  and  a  total  of  954  raters.  4  of 
which  did  not  provide  their  grade  level  on  the  background  form,  for  both  the  1976  study  and  the  1974  study  combined.  Many  of  the 
same  MET  raters  participated  in  both  rating  exercises. 
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APPENDIX  G:  DISTRIBUTION  OF  54  AFSC*  BY  CRADE 


58 


Distribution  of  Grades  by  54  AFStis  for  ll,)92-(iase 
Sample  Used  as  the  Base  for  Projections  of 
Grades  to  the  Non-Aircrew  Officer  Force 


DDL  GRADE OF  JOB 


AF SC 

Sampled 

Lieutenant 

Captain 

Major 

Lt  Colonel 

Colonel 

Total 

ooix 

17 

24 

41 

002X 

10 

54 

64 

003X 

8 

102 

110 

004X 

6 

37 

43 

005X 

17 

17 

34 

006X 

3 

27 

30 

007 X 

i 

28 

30 

59 

008X 

1 

7 

8 

009X 

2 

5 

7 

02 IX 

n 

9 

3 

23 

051 X 

l 

21 

7 

4 

33 

14XX 

272 

317 

313 

30 

932 

16XX 

22 

42 

20 

5 

11 

100 

17XX 

34 

100 

52 

19 

8 

213 

18XX 

200 

312 

150 

47 

16 

725 

20XX 

11 

40 

23 

16 

7 

97 

21XX 

1 

2 

3 

1 

1 

t 

22XX 

189 

190 

116 

9 

504 

23XX 

7 

21 

11 

9 

4 

52 

25XX 

17 

137 

50 

27 

10 

241 

26XX 

30 

106 

56 

16 

2 

210 

27XX 

43 

41 

84 

28XX 

77 

411 

198 

135 

7 

828 

29XX 

5 

46 

56 

59 

28 

194 

30XX 

32 

261 

193 

67 

25 

578 

31XX 

16 

54 

23 

13 

13 

119 

40XX 

29 

234 

153 

146 

43 

605 

46XX 

4 

65 

35 

21 

6 

131 

51XX 

24 

187 

90 

32 

9 

342 

55XX 

57 

131 

69 

59 

28 

344 

57XX 

5 

9 

9 

3 

3 

29 

60XX 

6 

70 

48 

26 

13 

163 

62XX 

3 

29 

13 

18 

4 

67 

63XX 

4 

16 

8 

6 

2 

36 

64XX 

19 

1)5 

69 

63 

18 

284 

65XX 

20 

83 

60 

46 

21 

230 

66XX 

1 

35 

50 

36 

8 

130 

67XX 

14 

71 

65 

26 

9 

185 

69XX 

3 

16 

20 

7 

3 

49 

70XX 

54 

248 

83 

65 

11 

461 

73XX 

24 

218 

106 

84 

10 

442 

74XX 

5 

37 

22 

25 

10 

99 

75XX 

3 

29 

35 

38 

15 

120 

79XX 

1 

31 

24 

15 

9 

80 

80XX 

22 

175 

89 

46 

23 

355 

59 


AFSC 

Sampled 

Lieutenant 

(Captain 

Major 

Lt  Colonel 

(Lionel 

Total 

81XX 

it 

78 

28 

27 

n 

155 

87XX 

4 

14 

3 

1 

22 

88XX 

1 

100 

51 

51 

18 

221 

80XX 

85 

30 

20 

21 

174 

ooxx 

18 

70 

30 

38 

18 

174 

OIXX 

10 

60 

22 

10 

0 

120 

02XX 

27 

61 

13 

6 

107 

07XX 

272 

273 

82 

33 

0 

<>60 

OOXX 

22 

11 

11 

0 

53 

Totals 

1,102 

4,584 

2.687 

1.061 

858 

11.102 
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APPENDIX  H:  MEANS  AND  STANDARD  DEVIATIONS 
OF  JOB  RATINGS 
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APPENDIX  I:  Sl'MMARY  OF  RELIABILITIES  FOK 
10  FACTORS,  MKT  Jl'DCMENT  OF  GRADE 
AND  PREDICTED  COMPOSITE  SCORE 


This  appendix  presents  the  inter-rater  reliabilities  (R^)  for  all  judgments  made  by  MET  raters  on 
10  factors  and  their  judgments  of  appropriate  grade  for  the  job.  Each  table  reports  sample  subset  R^r 's  in 
the  following  format  for  5  or  7  ratings  per  job  computed  by  the  Spearman-Brown  prophecy  formula. 


Policy 
Board  John 

Current 

Jobs 

Tout 

1974  Pilot 

N=  485 

(1) 

N=  1,687 

(4) 

N=  2,172 

(7) 

1976  Main  Study 

N=  1,240 

(2) 

N=  9,634 

(5) 

N=  10,874 

(8) 

Total  Sample 

N=  1,725 

(3) 

N  =  1 1 ,321 

(6) 

N-  13.046 

(9) 

Cells 

Description 

(1)  to  (3) 
(4)  to  (6) 
(7)  and  (8) 
(9) 


Policy  Board  jobs  used  as  the  criterion  in  computing 
validity  and  construction  of  the  grade  conversion  table 
Current  jobs  newly  collected  used  as  the  base  for 
making  projections  to  the  total  non-aircrew  force 
Rating  subset  information  used  for  reliability 
computations 

Grand  total  number  of  all  jobs  used  in  the  study 
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Inter-Rater  Reliabilities  iR^i  lor 
Faetor  I  formal  F.ib  alum3 


Job#  J"l# 


Hollo 

1  urrrni 

I  ot*l 

Holicv 

(  urrrni 

l.n.l 

Flint 

81 

81 

81 

Pilot 

88 

88 

88 

Mam 

8.1 

82 

82 

Pilot 

87 

87 

87 

Total 

87 

87 

87 

Total 

88 

87 

87 

k  for  average  of 

*i  rating* 

i*»-r  job 

k  tor  at  rragr 

of  "*  rating*  per 

Inter-Rater  Reliabilities  ( ^ )  fur 
Fat-tor  2.  Speeial  Training  anil  Work  F.xpeneiire*1 


74  .80  .80  Pilot  .81  .8.7  87 

80  .80  .80  Main  87  .87  87 

80  .80  .80  Total  87  87  87 


Ik  for  average  of  f»  ratings  per  job  k  f *»r  average  of  7  rating*  jmt  jot* 


Inter-Rater  Reliabilities  )  for 
Faetor  7.  W  orking  Camditions3 


Pilot 

.74 

.76 

,7i> 

Pilot 

.80 

.81 

HI 

Main 

.74 

.76 

.76 

Main 

.80 

.82 

HI 

Total 

.74 

.76 

.7!> 

Total 

80 

.82 

.81 

k  for  average  of 

ratings 

per  job 

k  for  average  of  7 

ratings  per  job 

Inter-Rater 

Reliabilities  (R^)  for 

Faetor  4.  Originality 

.  Ingenuity  and  Creativeness3 

Pilot 

.81 

.79 

.80 

Pilot 

.86 

.84 

87 

Main 

.74 

.78 

.78 

Main 

.84 

.87 

.87 

Total 

.80 

.78 

.79 

Total 

.87 

.87 

.81 

k  for  average  of  5 

ratings  per  job 

k  for  average  of  7 

ratings  per  job 

Inter-Rater  Reliabilities  (R^jJ  for 
Faetor  7.  Communication  Skills3 


Pilot  .82 

.79 

.81 

Pilot 

.87 

.84 

.87 

Main  .81 

.80 

.81 

Main 

.86 

.87 

.87 

Total  .82 

.80 

.81 

Total 

.86 

.87 

.87 

k  for  average  of  f>  ratings  per  job 
a(iomputed  by  the  Spearman- Brow 

n  prophecy  formula. 

k  for  average  of 

7  ratings  |»er  job 

IMot 

Main 

Total 
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Inter-Rater  Reliabilities  (R^)  for 
Factor  6.  Interpersonal  Skills* 


Pilot 

.78 

.76 

.77 

Pilot 

&3 

82 

.83 

Main 

.70 

.78 

.78 

Main 

.84 

.83 

.83 

Total 

.70 

.78 

.78 

Total 

.84 

83 

.83 

k  for 

avrragr  of  :»  ratings  prr  job 

k  for  avrragr  of 

7  ratings  prr  job 

Inter-Rater  Reliabilities  (R^)  for 
Factor  7.  Judgment  and  Decision  Making* 


Pilot 

.70 

.70 

.79 

Pilot 

.84 

.84 

.84 

Main 

80 

.79 

.79 

Main 

.85 

84 

.84 

Total 

.80 

.79 

.79 

Total 

.85 

.84 

84 

k  for  avrragr  of  5  ratings  prr  job  k  for  avrragr  of  7  ratings  prr  job 


Inter-Rater  Reliabilities  (R^)  for 
Factor  8.  Planning* 


ffl 

-> 

it 

1 


k  for  average  of  "»  ratings  prr  job  k  for  avrragr  of  7  ratings  prr  job 


Inter-Rater  Reliabilities  (Rll)  for 
Factor  9.  Management1* 


Pilot 

.79 

.83 

.82 

Pilot 

81 

87 

86 

Main 

.79 

.83 

.82 

Main 

.84 

87 

86 

Total 

.79 

.83 

.82 

Total 

84 

87 

86 

k  for  avrragr  of  !>  ratings  prr  job  k  for  avrragr  of  7  ratings  prr  job 

Inter-Rater  Reliabilities  (Rj^)  for 
Factor  10.  Risk* 


Pilot  .76 

.82 

.80 

Pilot 

.81 

86 

85 

Main  .76 

.82 

.81 

Main 

.8) 

86 

85 

Total  .76 

.82 

.80 

Total 

.81 

86 

85 

k  for  avrragr  of  5  ratings  prr  job 

k  for  avrragr  of  7 

ratings  prr  job 

af’.ompil!e<f  bv  the  Spearman-Brown  prophery  formula. 


.86  .86 

87  86 

87  86 


Pilot 

80 

.82 

.81 

Pilot 

Main 

.80 

.83 

.82 

Main 

Total 

.80 

.82 

.82 

Total 
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Inter-Rater  Reliabilities.  (R^l  for  MK  T 
Judgment  of  Vppropriate  tirade® 


r.i,*i 

.«<> 

.11 

.80  Pilot 

K) 

.18 

.12 

Mam 

.87 

>)2 

.')  1  Main 

.11 

.14 

.14 

j  Total 

t 

.87 

.12 

.01  Total 

,K) 

.14 

.1.8 

^  k  ft >r  JN»*rage 

fr' 

of  rating'  |M*r  jo! 

k  for  average  of  7 

rating*  per  job 

1 

». 

( 

Inter-Rater  Reliabilities  (Rii  )  for 

j 

Predicted  Composite  Score® 

Pilot 

.14 

.05 

.05  Pilot 

.% 

.17 

,<Mi 

Main 

<4 

.*15  Main 

.% 

.17 

.17 

1  otal 

.‘4 

.'Mi 

.15  Total 

.16 

.17 

.17 

k  for  average 

of  .*>  rating*  per  jol 

k  for  average  of  7 

ratings  per  job 

I  oitipulrii  lit 

.  the  SjH'armaM'Hrt 

>v*n  proplin 

>  formula 

I 

u 
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J:  Sl'M.MARY  OK  INTKK-HATKR  RKI.IABII.ITIKS 
KOK  COMPOSITK  SCORKS  BY  KATKR  TYPK  BASKII  ON 
SAM  1*1. K  Sl'BSKTS  OF  JOBS  WITH  TWO  OR  .WORK  RATKRS 


Remaining  N  Sets  When  Inter-Rater  Reliabilities 
( K k f°r  (Composite  Scores  are 
(Computed  for  Jobs  with  Two  or  More  Raters 


Jobs 

Jobs 

Policy 

(iurrenl 

Total 

Policy 

Current 

Total 

Total  Raters 

Officer  Raters 

Pilot 

m 

1,687 

2.172 

Pilot 

451 

1.043 

1.494 

Main 

1 .240 

9,634 

10.874 

Main 

914 

4.019 

4.933 

Total 

1.725 

11.321 

13,046 

Total 

1.365 

5.062 

6.427 

Enlisted  Raters 

Civilian  Raters 

Pilot 

474 

1,469 

1,943 

Pilot 

257 

441 

698 

Main 

1.240 

8,348 

9.588 

Main 

643 

3.759 

4.402 

Total 

1,714 

9,817 

11.531 

Total 

9<M! 

4.200 

5.100 

Inter-Rater  Reliabilities  (Ret. 

)  for  Predicted 

Composite  Scores  by  Total  and 

Rater  Type  for  Jobs  with  Two 

or  More  Raters3 

Jobs 

Jobs 

Policy 

Current 

Total 

Policy 

Current 

Total 

Total  Raters 

Officer  Raters 

Pilot 

.94 

.95 

.95 

Pilot 

.96 

.96 

.96 

Main 

.94 

.96 

.95 

Main 

.95 

.97 

.96 

Total 

.94 

.96 

.95 

Total 

.95 

.97 

.96 

foliated  Raters 

Civilian  Raters 

Pilot 

.94 

.95 

.94 

Pilot 

.94 

.96 

.95 

Main 

.94 

.95 

.95 

Main 

.94 

.96 

.96 

Total 

.94 

.95 

.95 

Total 

.94 

.96 

.96 

k  for  average  of  5  ratings  per  job  k  for  average  of  5  ratings  per  job 

Viomputetf  by  the  Spearman-Brown  prophery  formula 
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Inter-Rater  Reliabilities  (R|^)  for  Predicted 
Composite  Scores  by  Total  and 
Rater  Type  for  Jobs  with  Two  or  More  Raters® 


Jobs 

Jobs 

Policy 

Current 

Total 

Policy 

Current 

Total 

Total  Raters 

Officer  Raters 

Pilot 

.96 

.97 

.96 

Pilot 

.97 

.97 

.97 

Main 

.% 

.97 

.97 

Main 

.96 

.98 

.97 

Total 

.96 

.97 

.97 

Total 

.97 

.98 

.97 

Enlisted  Raters 

Civilian  Raters 

Pilot 

.96 

.96 

.96 

Pilot 

.96 

.97 

.97 

Main 

.95 

.96 

.96 

Main 

.96 

.97 

.97 

Total 

.% 

.96 

.96 

Total 

.96 

.97 

.97 

k  for  average  of  7  ratings  per  job  k  for  average  of  7  ratings  per  job 

Computed  by  the  Spearman -Brown  prophecy  formula 
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«o-«ofj  tot 

AlR  FORCE  HUMAN  RESOURCES  LA*  OROOKS  AF*  TX 
HAnA&CHEnT  CNilNEERINS  Tj AN  APPLICATION  OF  OFFICER 
DEC  «0  K  FInSTUEN,  6  N  mATThEnS,  m  h  pope 

F/S 

6RA0C  RcfluIR. 

UNCLASSIFIED 

AFHRL-TR-iQ.ja 

NL 

SUPPLEMENTARY 

> 


INFORMATION 


„/h9357o'x' 
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department  op  the  air  force 

AIR  FORCE  HUMAN  RESOURCES  LABORATORY  ( AFSC ) 
BROOKS  AIR  FORCE  BASE.  TEXAS  7823S 


MMT  to 
ATTN  Of: 

MUCCT: 


TSR  ^ 

Removal  of  Export  Control  Statement 


1 6  JAM  1981 


to.-  Defense  Technical  Information  Center 
Attn:  DT1C/DDA  (Mrs  Crumbacker) 

Cameron  Station 
Alexandria  VA  22314 

1.  Please  remove  the  Export  Control  Statement  which  erroneously  appears  on 
the  Notice  Page  of  the  reports  listed  MBtMHaE&HMSlft.  This  statement  is 
Intended  for  application  to  Statement  B  reports  only. 

2.  Please  direct  any  auestions  to  AFHRL/TSR,  AUTOVON  240-3877. 

FOR  THE  COMMANDER 

WENDELL  L.  ANDERSON,  Lt  Col,  USAF 
Chief,  Technical  Services  Division 


1  Atch 

List  of  Reports 


Cy  to:  AFHRL/TSE 


